History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: IDEA-14955
Type: Performance Problem Performance Problem
Status: Open Open
Assignee: Eugene Zhuravlev
Reporter: Tim Kuchlein
Votes: 2
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
IDEA: Feedback

GUI hangs after ant build

Created: 14 Sep 07 21:22   Updated: 21 Nov 07 00:24
Component/s: Ant Integration

File Attachments: 1. Zip Archive 7126_rob_17.09.2007_14.11.54.zip (2.78 Mb)
2. Zip Archive 7126_tim_12.10.2007_12.57.04.zip (2.54 Mb)
3. Zip Archive 7274_tim_25.09.2007_10.09.49.zip (2.99 Mb)
4. Zip Archive 7294_tim_25.09.2007_10.48.35.zip (2.55 Mb)
5. Text File idea_14955_stacktrace.log (674 kb)
6. File Info.plist (3 kb)

Environment: OS X 10.4, Mac Pro 2.66GHz processors, 4GB RAM, Apple JDK 1.5

Build: 7,274
Severity: High


 Description  « Hide
We have a project with in the region of 50 and growing modules which is being developed on OS X using IntelliJ Selena (various builds including M2) and we're seeing a problem that when the ant build completes the GUI is unresponsive for up to 2 minutes afterwards. The actual build only takes 15 seconds. We haven't been able to work out what it doing beyond sitting there hammering one of the CPU's. During the build we see both Java and IntelliJ using a lot of CPU as expected but once the Java process is complete IntelliJ continues to use the CPU and doesn't appear to be doing anything.

Is there something we can look for or at to help resolve this? I have tried 6.0.5 and builds 7051, 7087, 7-0M2 and 7274 all of which exhibit the same issue. When I put the entire project into a ram disk it was quite a bit faster but even in this case the GUI was still unresponsive after the Ant build was complete. While the ram disk solution eases the issue somewhat we are going to be adding quite a few more modules in the future and we're worried that it will continue to get worse as time goes one. Has anyone else seen this issue?



 All   Comments   Work Log   Change History      Sort Order:
Eugene Zhuravlev - 17 Sep 07 16:37
Hi Tim,
Could you please record a CPU snapshot and attach it to the issue. Also a simple thread dump taken right during the "freeze" may help.

Tim Kuchlein - 17 Sep 07 23:29
CPU usage as requested

Tim Kuchlein - 17 Sep 07 23:30
Thread Dump as requested

Eugene Zhuravlev - 18 Sep 07 12:37
Thanks Tim.
From the thread dump I see that there are lots of OutOfMemory errors. This might be caused by lengthy output that does not fit into the currently available java heap space. Please check that "Settings | General | Limit run/debug consoles output" option is activated. Also check the heap size that IDEA is configured to use and increase it if neccesary.

Tim Kuchlein - 18 Sep 07 18:51
Thanks for taking a look at the log. I did try what you suggested by changing it from 1024Kb to 4096Kb but it didn't have any effect on the build times using build 7274. The VMOptions that I'm passing into IntelliJ are:

-Xms16m -Xmx1024m -XX:MaxPermSize=512m -Xbootclasspath/p:../lib/boot.jar -ea -agentlib:yjpagent

Which I think should be sufficient to get a build done. Also one thing I noticed today is that when I'm viewing the build progress with tree mode rather than text mode it takes about 3 times longer to build. Do you have any other suggestions or things we can try? Do you actually know what IntelliJ is doing when its stuck in the GUI thread? Is there a way to find out?

I know that version 7 is still beta but we've seen many other issues with subversion checkins (missed files) and doing global searches in the project not finding everything it should. Should we open additional Jira issues for these or add these problems to this issue?


Tim Kuchlein - 24 Sep 07 19:00
Do you have any update on this issue? Thanks.

Eugene Zhuravlev - 25 Sep 07 14:17
Hi Tim,
First, do you still find OutOfMemory errors in your log?

> -Xms16m -Xmx1024m -XX:MaxPermSize=512m

The values are too big.... The effect is the opposite from what you might expect - the GUI gets frozen for lengthy GC-pauses since there are plenty of RAM to hold the generated garbage Try to set -Xmx256m and -XXMaxPermSize=100m (not more). The gui should behave snappier now.
I myself use -Xmx256m for IDEA project which is relatively big, and it works fine for me.

As for the tree mode - you are right - the problem here is in too many "rebuild" events that are generated when new node is inserted into the tree. I agree that this is not so optimal as could be and we definitely will do something with this.

By the way, how may output does your build produce? (Just truing to set up something similar for the testcase).


Eugene Zhuravlev - 25 Sep 07 14:25
By the way, did you try the latest EAP? It contains one small thing that should make tree expansion faster.

Tim Kuchlein - 25 Sep 07 18:17
I tried what you suggested and the build now takes longer (at least the ant task). From IntelliJ:

published ivy to /Users/tim/IdeaProjects/platform/com.txcore.test.workspace/trunk/../../ivy-repo/txcore/com.txcore.test.workspace/0.0.1-SNAPSHOT/ivys/ivy-0.0.1-SNAPSHOT.xml

Ant build completed successfully in 45s at 10:01:40 AM

where as I got the following from running the ant task on the command line:

[ivy:publish] published ivy to /Users/tim/IdeaProjects/platform/com.txcore.test.workspace/trunk/../../ivy-repo/txcore/com.txcore.test.workspace/0.0.1-SNAPSHOT/ivys/ivy-0.0.1-SNAPSHOT.xml

BUILD SUCCESSFUL
Total time: 18 seconds

It took a full minute in addition to the ant build time before I could use the GUI again which is the problem that I had to begin with. Is there anyway I can see if IntelliJ is going a bunch of GC's one after the other? A single GC shouldn't take anywhere near 1 minute to run I would think.

I'm still running 7274 but will try the latest, 7294, to see if that is any better. The ant task generates 6586 lines of output but we're all OK with using just the text output.


Tim Kuchlein - 25 Sep 07 18:18
The latest log file. I don't know how to look at the contents myself so I can't say if we're still seeing OOM errors or not.

Tim Kuchlein - 25 Sep 07 18:52
I've included a log from 7294 in case the difference in versions can help you.

Tim Kuchlein - 02 Oct 07 19:33
Any update on this issue? Its gotten to the point where we've given up on using IntelliJ to build our project since its too slow and we've switched to the command line for Subversion check-in/outs due to it missing files. If we can't get these issues resolved we are going to have to start looking at other options.

Colin Fleming - 03 Oct 07 09:25
If this issue is really caused by excessive logging during the build, it could at least be helped by IDEADEV-11517 and IDEADEV-20623. Unfortunately these have both been marked as not to be fixed in Selena. Can this be changed?

Tim Kuchlein - 11 Oct 07 17:44
Its been over 2 weeks since we've seen any update on this issue. This is a fairly serious problem for us since it impacts our productivity not to mention causing quite a bit of frustration. Honestly I was expecting better support from a product like IntelliJ. This is a show stopper issue for us and if we can't get this issue resolved we'll have to look at other products. Surely this problem isn't that hard to fix?

Maxim Mossienko - 11 Oct 07 18:16 - edited
AFAIU:
First line in idea log file is:
rob:~ rob$ sh /Applications/IntelliJ\ IDEA\ 7-0M2.app/bin/idea.sh
/Applications/IntelliJ IDEA 7-0M2.app/bin/idea.sh: line 1: /Applications/IntelliJ IDEA 7-0M2.app/bin/../bin/idea.vmoptions: No such file or directory

Given that, IDEA starts with default VM heap size 64M and quickly goes out of heap (default VM size in IDEA config is 190M).


Tim Kuchlein - 11 Oct 07 18:57
You are correct that its missing this file but it would appear that its not shipped with IntelliJ by default. Also this was something we ran back in the middle of September and we've tried several versions since then with the same behaviour so I can't see why this would be the cause of the issue. Also AFAIK idea.sh isn't used to start IntelliJ on a Mac but uses a file called Info.plist (attached). The two files you should look at have "tim" in the name. Let me know if there is anything else we can try at this end.

Tim Kuchlein - 11 Oct 07 18:58
Mac script to run Selena 7294.

Maxim Mossienko - 11 Oct 07 19:15
Ok, so you tried to increase vm size without luck?

Tim Kuchlein - 11 Oct 07 21:19
Assuming that the JVM settings are in Info.plist then yes. We have also tried several different builds of Selena and even version 6.

Maxim Mossienko - 12 Oct 07 18:16
AFAIK, when you run via *.sh Info.plist is not used so you need to create that file with (e.g.) following content or launch in Mac specific way
(via open or desktop icon):
-Xmx256m
-XX:MaxPermSize=129m
-ea

Tim Kuchlein - 12 Oct 07 19:09
I've tried making the changes that you suggested and the build times are about twice as long but the wait time is about 2/3. I don't understand why the build time should be any longer since we're not changing the ant task. Its jumped from 18s to 33s. Over all the time is better but still not where it needs to be. We should have virtually no wait time following the compile and nor should the compile be much longer than running the build from the command line. We're seeing a high CPU load but almost no disk activity which suggests that its stuck in the IDE somewhere. Where do we go here.

Maxim Mossienko - 12 Oct 07 19:35
Please, attach thread dump and snapshot from latest IDEA 7 RC version

Tim Kuchlein - 12 Oct 07 21:20
CPU trace of 7 M2

Maxim Mossienko - 12 Oct 07 22:39
Please, use IDEA version 7 RC 6 from here
http://www.jetbrains.net/confluence/display/IDEADEV/Selena+EAP

Tim Kuchlein - 08 Nov 07 18:51
After fighting with this for some time we have re-written our build into Java using Leafcutter and this has solved this issue for us. The build times when nothing has changed has dropped from about 75 seconds (15-20 to build and 55-60 seconds with the IDE locked up) to 4-5 seconds total and the IDE never locks up. Even when we do a full build which takes 3 min 18 sec the IDE is usable the whole time.

Feel free to close this issue.


Colin Fleming - 09 Nov 07 15:43 - edited
Please do not close this issue.

I rewrote the build in Java, it worked fine until this morning. I added some new functionality, and now the build completely locks IDEA. Since the build is in Java, we're using a Run configuration to start it. The build starts, and very quickly IDEA locks totally. I can't do anything with the GUI, the only thing I can do is kill the process. I can't even get a thread dump. If I use StackTrace, it returns:

1339
Signature mismatch: 55 89 e5 57 56 53 e8 b5 1b e 0 8b 7d 8 8b 75 c 8b 55 10 8b 83 de b8

Using kill -QUIT pid on the IDEA process causes nothing to come out in Console. I don't know how to get a CPU snapshot on a non-EAP version.

As far as I can tell, the build process completes normally, I can see it pegging one CPU for about 3 minutes then it stops. I can't see any output, of course.

Please advise how to debug this. This is pretty urgent for us.


Taras Tielkes - 09 Nov 07 20:24
Colin,

You could try "jps"/"jstack" tools from JDK to obtain a stacktrace.

To obtain a CPU snapshot from a non-EAP IDEA build you need to enable the profiler: add "-agentlib:yjpagent" in idea.vmoptions.exe.
If the profiler is enabled but IDEA is too unresponsive to click the "CPU snapshot" button, try to download an evaluation version of YourKit, and connect using the YourKit client - and then capture a CPU snapshot using that.


Colin Fleming - 12 Nov 07 22:35
I haven't had time to play around with this much today, but I just tried it now. IntelliJ completely locked up as soon as I pressed the green run arrow - it didn't even start the build. Same symptoms - thrashing a single CPU, completely unresponsive. I tried jstack and got the following error:

~/dev/platform> jstack 5056
Attaching to process ID 5056, please wait...
Error attaching to process: Error attaching to process, or no such process

I can't use the built in profiling since the IDE locks completely, and I've already used a YourKit eval license. I don't want to buy it just to diagnose a problem with IDEA. Any more suggestions?


Eugene Zhuravlev - 12 Nov 07 22:54
Colin, if IDEA unlocks at some moment, you could have started profiling before pressing "Run" and stopped it after IDEA got unlocked.
If IDEA locks "forever", the only way is to restart it with console (use startup script instead of binary launcher) and take several thread dumps in the console window when IDEA is locked (you don't need jstack for that and it works 100%).

Colin Fleming - 21 Nov 07 00:24
Sorry for the delay with this, I've been very busy. The problem is not with my build at all. This only happens on my machine, and only with 7.0.1. I'm mostly using 7.0-M2, it works great. The Selena EAP also works, although I'm not using it because of another bug. I'd like to use 7.0.1, but apart from this stalling bug it's also unuseably slow. Any ideas where I should start to look to diagnose this problem?