[#IDEADEV-15600] Code formatter behaves weird if custom language uses own whitespace token(s)

IDEA: Development

Code formatter behaves weird if custom language uses own whitespace token(s)

Created: 23 Mar 07 20:01 Updated: 03 Oct 07 01:29

Component/s:

Plugin Support. API

Fix Version/s:

None

Original Estimate:

Unknown

Remaining Estimate:

Unknown

Time Spent:

Unknown

Build:	6,775
Severity:	Medium

Description

« Hide

The formatting of custom languages does not work properly if the language defines its own set of whitespace tokens instead of using TokenType.WHITE_SPACE. The result is a very odd behavior of the formatter that is extremely hard to track down, especially because the language API (com.intellij.lang.ParserDefinition#getWhitespaceTokens) tricks one into thinking that using custom whitespace tokens is supported.

This should at least be very well documented (both in the formatter and the ParserDefinition) - I just spent a night trying to get some simple indentation to work until I found out that TokenType.WHITE_SPACE has a special meaning to the code formatter.

All

Comments

Work Log

Change History

Sort Order:

[ Permlink | « Hide ]

Sascha Weinreuter - 02 Oct 07 01:39

Max, I don't think this necessarily needs to be fixed. Just add a sentence in the ParserDefinition's and Formatter's Javadoc that it it's strongly recommended to use TokenType.WHITE_SPACE if one wants to implement a custom formatter without getting a headache

[ Permlink | « Hide ]

Maxim Shafirov - 03 Oct 07 00:11

Thanks for reminder. I've just updated javadoc in ParserDefinition but I'm not quite sure what Formatter do you mean?

[ Permlink | « Hide ]

Sascha Weinreuter - 03 Oct 07 01:29

Sorry, I meant com.intellij.formatting.FormattingModelBuilder (and/or com.intellij.lang.Language#getFormattingModelBuilder). Although there's no direct relationship with whitespace handling here, adding some note like "For certain aspects of the custom formatting to work properly, it is recommended to use TokenType.WHITE_SPACE as the language's whitespace tokens. @see ParserDefinition" wouldn't hurt.