In the case a session hangs because of a problem somewhere, one should interrupt PROOF on the client (possibly exiting and re-starting the ROOT session), reset the PROOF session and have a look at the session logs.
Starting from ROOT v5.24/00, a graphic interface to this method is available via the static TProof::LogViewer . This static method opens a frame where you can enter the URL of the cluster and the relative session number, and to choose which logs to retrieve and display. The log frame open in this way is the same that, starting with version 5.20/00, is obtained by pushing the buttong "Show Logs" in the PROOF dialog box during or at the end of running.
As an example, entering
root  TProof::LogViewer("firstname.lastname@example.org")
the following window should pop up:
On the top left you can change the master URL (you need to push the 'Get logs info' for making that effective) and choose the session from which you want the logs: '0' means the last one (default), negative numbers indicate the relative position wrt the last one (e.g. use '-1' to get the next to last one).
On the bottom left of the frame you can choose from which node to get the logs. Once you have made your choice you have to push 'Display' to get the last 100 lines of each chose log displayed in the box. You can set the number of lines to display (check 'all' to get all what available) on the bottom control bar.
Note the PROOF generates some service messages (e.g. for memory monitoring) which by default are always filtered-out from the logs (even if you have checked 'all'). You should also check 'svcmsg' to get really everything.
Note also that for large logs retrieval and displaying may be quite slow.
Browsing logs from the command line
In the case graphics is not available or too slow, the logs can be browsed directly in the session window. The PROOF session logs for the last session can be retrieved from the master and workers via:
root  TProofLog *pl = TProof::Mgr("master")->GetSessionLogs()The TProofLog class contains methods to browse and search the logs. To display the last 10 lines of each log file just type
root  pl->Display()
this typically allows to identify the node(s) where the problem is. The full log for one node, e.g. "0.3", can be displayed specifying the node ordinal and the starting line
root  pl->Display("0.3",0)
If it is not clear which node(s) had problems, one can try by searching for a specific text or keywords, e.g.
root  pl->Grep("violation")
the list of occurencies of "violation" will be displayed, with details about the node ordinal numer (e.g. "0.5") and the line(s) where it occured (e.g. 23); one can then browse directly the related portion of the log with
root  pl->Display("0.5", 20, 100)
which will display 100 lines from the log of node "0.5", starting from line 20.
The logs can be saved into a file with
root  pl->Save("0.7", "file_with_logs_of_worker_0.7.txt") root  pl->Save("*", "file_with_all_logs.txt")