1.) Create new class.
2.) Make sure it's access modifier is default access.
3.) Generate constructor by Alt+Ins.
4.) Constructor will have public access modifier.
Same thing happens with inner classes.
It would be nice for constructor to get more reasonable modifier (maybe just copy class access modifier).
This behavior is observed also in 6.0.* versions of IDEA.