Re: parallel build does not work

Hi Tom,

   I've a MacPro 8-core and a Linux 24-core at work, and a MacPro 4-core at home. On all these machines:

./configure
make -j 8 | 24 | 4

works fine. Having a correct parallel Makefile is primordial and a top priority in the multi-core era. As for as I see it works fine for the default setup. Can you try the svn trunk and just run default ./configure and make -j 4. That should work. If not it must be something with your setup.

Do you write to remote/afs disks?

Cheers, Fons.

Tom Roberts wrote:
> Fons:
>
> That is the make I have, too. While it says powerpc in the last line,
> /usr/bin/make is a universal binary for both i386 and ppc7400. It's from
> Xcode version 3.1.3.
>
> BIG DIFFERENCE: I am doing a 32-bit build, so must give configure the
> architecture macosx (by default it does a 64-bit build).
>
> When I just do "make -j 4", it fails. Ditto when I add all those
> --disable-*. The fact that the make fails in different ways suggests to
> me that the makefiles are not really parallel-safe. It may be that your
> 8 CPUs just happen to work but my 4 CPUs fail. Can you try with 4?
>
> (un-tar a fresh source)
> ./configure macosx
> make -j 4
>
> It is also likely that your build machine has some optional stuff
> installed that I do not, so perhaps you should add all those --disable-*
> to configure.
>
> In one failure, which did not resolve by re-running "make -j 1", some .c
> or .cxx file which was clearly constructed programmatically just ended
> in the middle, and of course failed to compile. I believe its name began
> G__ but did not take notes. Constructing a file is, of course, the sort
> of thing that may need a lock file to be parallel-safe.
>
> Tom Roberts
>
>
>
> Fons Rademakers wrote:

>> Hi Tom,
>>
>>  my main ROOT dev platform is a MacPro 8-core and make -j 8 work fine. 
>> To do the static build do:
>>
>> make -j 8
>> make static
>>
>> Which make do you use? A non-Apple version from Fink? I use:
>>
>> (proof) [144] which make
>> /usr/bin/make
>> (proof) [145] make -v
>> GNU Make 3.81
>> Copyright (C) 2006  Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions.
>> There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
>> PARTICULAR PURPOSE.
>>
>> This program built for powerpc-apple-darwin9.0
>>
>>
>> Cheers, Fons.
>>
>>
>>
>> Tom Roberts wrote:
>>> This is root 5.24 on Mac OS X 10.5.8 (the latest Leopard). This is a 
>>> Mac Pro with 4 cores.
>>>
>>> Is this a known problem? Is there any fix? -- building Root now 
>>> dominates my build time, and a 4-CPU build would help substantially. 
>>> "make -j 1" works fine, but "make -j 4" fails in different ways each 
>>> time.
>>>
>>>
>>> Tom Roberts
>>>
>>>
>>> Here are the details:
>>>
>>> I need to link with Root only to read/write TNtuple-s from/to 
>>> TFile-s, so I disable everything I can get away with (I got the list 
>>> from configure --help and pruned it until it built). I use static 
>>> libraries to avoid shared-library headaches, and to keep the binary 
>>> installation package as small as possible. I will be building on 
>>> Linux and Windows Xp as well, including a Linux supercomputer that 
>>> does not permit shared objects; I'm doing Mac OS X first.
>>>
>>> With "make -j 1", Root builds and my program builds and runs 
>>> correctly. But with "make -j 4" I usually don't get to the "make 
>>> static".
>>>
>>> This is 5.24. In the past I have noticed similar behavior with Root 
>>> 5.20, and just lived with "-j 1".
>>>
>>> I downloaded the 5.24.00 source on 2009/08/18 and unpacked it from 
>>> /long/path
>>>
>>> I then did:
>>>   ROOTARCH=maxcosx # 32-bit build
>>>   cd /long/path
>>>   mv root root-5.24
>>>   cd root-5.24
>>>   unset MAKEFLAGS
>>>   # disable all but: cint5, zlib
>>>   ./configure >>build.log 2>&1 $ROOTARCH \
>>>   --disable-afs --disable-alien --disable-asimage \
>>>   --disable-astiff --disable-bonjour --disable-builtin-afterimage \
>>>   --disable-builtin-ftgl --disable-builtin-freetype \
>>>   --disable-builtin-glew --disable-builtin-pcre --disable-castor \
>>>   --disable-chirp --disable-cint7 --disable-cintex \
>>>   --disable-clarens --disable-dcache --disable-exceptions \
>>>   --disable-explicitlink --disable-fftw3 --disable-gdml \
>>>   --disable-gfal --disable-g4root --disable-globus --disable-glite \
>>>   --disable-gsl-shared --disable-krb5 --disable-ldap \
>>>   --disable-genvector --disable-mathmore --disable-memstat \
>>>   --disable-minuit2 --disable-monalisa --disable-mysql \
>>>   --disable-odbc --disable-opengl --disable-oracle \
>>>   --disable-pch --disable-peac --disable-pgsql --disable-pythia6 \
>>>   --disable-pythia8 --disable-python --disable-qt \
>>>   --disable-qtgsi --disable-reflex --disable-rfio \
>>>   --disable-roofit --disable-rpath --disable-ruby \
>>>   --disable-sapdb --disable-shadowpw --disable-shared \
>>>   --disable-soversion --disable-srp --disable-ssl \
>>>   --disable-table --disable-unuran --disable-winrtdebug \
>>>   --disable-xml --disable-xrootd --disable-xft
>>>   make -j 4 >>build.log 2>&1
>>>
>>> The make dies in different ways each time (after removing the 
>>> previous attempt, re-unpacking the source, and re-doing the above 
>>> ./configure). Re-doing the make sometimes completes, sometimes doesn't.
>>>
>>> Here's the tail of build.log for the most recent failure:
>>> Generating dictionary hist/histpainter/src/G__HistPainter.cxx...
>>> core/utils/src/rootcint_tmp -cint -f 
>>> hist/histpainter/src/G__HistPainter.cxx -c 
>>> hist/histpainter/inc/TGraph2DPainter.h 
>>> hist/histpainter/inc/TGraphPainter.h 
>>> hist/histpainter/inc/THistPainter.h hist/histpainter/inc/TLego.h 
>>> hist/histpainter/inc/TPainter3dAlgorithms.h 
>>> hist/histpainter/inc/TPaletteAxis.h hist/histpainter/inc/LinkDef.h
>>> Error: cannot open file "sstream"  include/Math/Util.h:18:
>>> Generating dictionary tree/treeplayer/src/G__TreePlayer.cxx...
>>> core/utils/src/rootcint_tmp -cint -f 
>>> tree/treeplayer/src/G__TreePlayer.cxx -c 
>>> tree/treeplayer/inc/TBranchProxy.h 
>>> tree/treeplayer/inc/TBranchProxyClassDescriptor.h 
>>> tree/treeplayer/inc/TBranchProxyDescriptor.h 
>>> tree/treeplayer/inc/TBranchProxyDirector.h 
>>> tree/treeplayer/inc/TBranchProxyTemplate.h 
>>> tree/treeplayer/inc/TChainIndex.h tree/treeplayer/inc/TFileDrawMap.h 
>>> tree/treeplayer/inc/TFormLeafInfo.h 
>>> tree/treeplayer/inc/TFormLeafInfoReference.h 
>>> tree/treeplayer/inc/TFriendProxy.h 
>>> tree/treeplayer/inc/TFriendProxyDescriptor.h 
>>> tree/treeplayer/inc/TRefArrayProxy.h tree/treeplayer/inc/TRefProxy.h 
>>> tree/treeplayer/inc/TSelectorDraw.h 
>>> tree/treeplayer/inc/TSelectorEntries.h 
>>> tree/treeplayer/inc/TTreeDrawArgsParser.h 
>>> tree/treeplayer/inc/TTreeFormula.h 
>>> tree/treeplayer/inc/TTreeFormulaManager.h 
>>> tree/treeplayer/inc/TTreeIndex.h tree/treeplayer/inc/TTreePlayer.h 
>>> tree/treeplayer/inc/TTreeProxyGenerator.h 
>>> tree/treeplayer/inc/TTreeTableInterface.h tree/treeplayer/inc/LinkDef.h
>>> Warning: Error occurred during reading source files
>>> Generating dictionary gui/ged/src/G__Ged.cxx...
>>> core/utils/src/rootcint_tmp -cint -f gui/ged/src/G__Ged.cxx -c 
>>> gui/ged/inc/HelpSMText.h gui/ged/inc/TArrowEditor.h 
>>> gui/ged/inc/TAttFillEditor.h gui/ged/inc/TAttLineEditor.h 
>>> gui/ged/inc/TAttMarkerEditor.h gui/ged/inc/TAttTextEditor.h 
>>> gui/ged/inc/TAxisEditor.h gui/ged/inc/TCurlyArcEditor.h 
>>> gui/ged/inc/TCurlyLineEditor.h gui/ged/inc/TF1Editor.h 
>>> gui/ged/inc/TFrameEditor.h gui/ged/inc/TFunctionParametersDialog.h 
>>> gui/ged/inc/TGedEditor.h gui/ged/inc/TGedFrame.h 
>>> gui/ged/inc/TGedMarkerSelect.h gui/ged/inc/TGedPatternSelect.h 
>>> gui/ged/inc/TGraphEditor.h gui/ged/inc/TH1Editor.h 
>>> gui/ged/inc/TH2Editor.h gui/ged/inc/TLineEditor.h 
>>> gui/ged/inc/TPadEditor.h gui/ged/inc/TPaveStatsEditor.h 
>>> gui/ged/inc/TPieEditor.h gui/ged/inc/TPieSliceEditor.h 
>>> gui/ged/inc/TStyleDialog.h gui/ged/inc/TStyleManager.h 
>>> gui/ged/inc/TStylePreview.h gui/ged/inc/LinkDef.h
>>> make[2]: *** [hist/hist/src/G__Hist.cxx] Bus error
>>> make[2]: *** Deleting file `hist/hist/src/G__Hist.cxx'
>>> make[2]: *** Waiting for unfinished jobs....
>>>
>>> In this case re-doing the "make -j 4" happened to succeed. So I 
>>> continued:
>>>   make -j 1 static
>>>   ... success
>>>   cd /path/to/my/source
>>>   make
>>>   ... success
>>>   cd test
>>>   ./all
>>>   ... 72 tests run
>>>   All Tests Passed
>>>
>>> Several of those tests require Root I/O of TNtuples.
>>>
>>> When I do "otool -L g4beamline", the only dylib-s it references are 
>>> in /usr/X11/lib and /usr/lib, plus /sw/lib/libXm.2.dylib (I include 
>>> this last with my binary distribution).
>>>
>>

-- 
Org:    CERN, European Laboratory for Particle Physics.
Mail:   1211 Geneve 23, Switzerland
E-Mail: Fons.Rademakers_at_cern.ch              Phone: +41 22 7679248
WWW:    http://fons.rademakers.org           Fax:   +41 22 7669640