Error building libsmm

[Alfio Lazzaro]

OK, so it is a problem at linking time (the *.o files are fine).
It seems that for a strange reason it is compiling for MIC, while it should 
ignore the MIC directives inside the files...
Therefore it is a compiler issue (this problem doesn't exist for v14). 
Browsing in the web, I saw that there is a flag:

–no-offload

Could you try in that? If it works then I will ask CP2K developers to add 
the flag into repository.
Sorry about that...

Alfio



Il giorno mercoledì 29 ottobre 2014 23:25:49 UTC+1, Abhishek Bagusetty ha 
scritto:
>
> I am using linux.intel, the PBS engine is fine and -openmp flag is used 
> during compilation.
> The issue here is, all the jobs get terminated with an error report from 
> PBS script stating these details : 
>
>
>
> *ifort : warning # : Environment configuration problem encountered. Please 
> check MPSS installation and environment setup. (lots of these 
> warnings)ifort : warning # : -liomp5 linked in dynamically, static library 
> not available for Intel (R) MIC architecture.-loffload linked in 
> dynamically, static library not available*
>
> No *.x files were found in /run_tiny_dnn/output_linux.intel/. The only 
> files in output dir are *.o and some *MIC.o. 
> linkers (liomp5, loffload) above are specific to MICs and I am not sure 
> why these flags came in during compilation. I have check linux.intel, 
> config.in, generate.bash, generate files to find the source but in vain. 
> Was not able to find any -mmic flags or offload flags that could have 
> triggered this error. 
>
> ifort (13.0.0v)
> Let me know if there is any discrepancy that could have occurred. 
>
> Abhishek
> On Wednesday, October 29, 2014 4:53:42 AM UTC-4, Alfio Lazzaro wrote:
>>
>> Yes, 2.6 has a different way to run the libsmm kernels based on OpenMP. 
>> The files under config are update for that. Could you check if your pbs is 
>> still working? 
>> another important thing is that you should have "-openmp" in your file 
>> config/linux.intel under the variable:
>>
>> target_compile
>>
>> Then, do you have any output for errors? We need to understand where the 
>> code doesn't work.
>> Could you check if it is producing the *.o and *.x files under 
>> /run_tiny_dnn/output_linux.intel/? If not, then it means that the libsmm 
>> procedure has not started at all...
>>
>> Alfio
>>
>>
>>
>>
>> Il giorno mercoledì 29 ottobre 2014 03:48:46 UTC+1, Abhishek Bagusetty ha 
>> scritto:
>>>
>>> Hi Alfio,
>>>
>>> I am trying to build the libsmm in cp2k-2.6 and it looks like there has 
>>> been some recent commits on that. I am trying to build the libsmm and when 
>>> I run the tiny1, the pbs jobs run but there are no output files (*.out) 
>>> generated in the /run_tiny_dnn/output_linux.intel/. It looks something is 
>>> wrong with my setup. I haven't changed any file. Do you have any pointers 
>>> that comes to the rescue. 
>>>
>>> Thanks for your time,
>>>
>>> Abhishek
>>>
>>> On Thursday, October 23, 2014 4:23:20 AM UTC-4, Alfio Lazzaro wrote:
>>>>
>>>> Probably I was too optimistic when I asked you to replace the gfortran 
>>>> with ifort...
>>>> Let me explain the problem. The errors are due to the construct:
>>>>
>>>> ERROR STOP
>>>>
>>>> which is inside the file multrec_gen.f90 (lines 288 and 366). This is a 
>>>> 2008 Fortran construct and it appears that neither your ifort nor gfortran 
>>>> versions are supporting it.
>>>> Then there is no way to compile it unless we change the code. I would 
>>>> ask you to restore gfortran as host compiler in the config/linux.intel, and 
>>>> then open the file multrec_gen.f90 and remove the keyword "ERROR" (keep 
>>>> STOP) in the two lines. Try this and see if it fixes the problem.
>>>>
>>>> I see also that "ERROR STOP" is used once in the file generate.bash 
>>>> file. You can remove from there too.
>>>>
>>>> Alfio
>>>>
>>>>
>>>>
>>>> Il giorno giovedì 23 ottobre 2014 06:54:24 UTC+2, Abhishek Bagusetty ha 
>>>> scritto:
>>>>>
>>>>> Thanks for figuring out the issue. I have used ifort instead of 
>>>>> gfortran and I still get a different set of errors. 
>>>>>
>>>>> During the 3rd step  ./generate -c config/linux.intel -j 20 -t 16 -w 
>>>>> pbs small1
>>>>>
>>>>> These are the set of errors which looks different to the earlier ones :
>>>>>
>>>>> multrec_gen.f90(267): error #5144: Invalid character_kind_parameter. 
>>>>> No underscore
>>>>>      IF (ANY(best_square<1)) ERROR STOP "tiny opts file needs 
>>>>> sufficiently many square sizes"
>>>>>
>>>>> ---------------------------------------------------------------------------------------------^
>>>>> multrec_gen.f90(267): error #5082: Syntax error, found IDENTIFIER 
>>>>> 'STOP' when expecting one of: ( % : . = =>
>>>>>      IF (ANY(best_square<1)) ERROR STOP "tiny opts file needs 
>>>>> sufficiently many square sizes"
>>>>> -----------------------------------^
>>>>> multrec_gen.f90(331): error #5144: Invalid character_kind_parameter. 
>>>>> No underscore
>>>>>        ERROR STOP "MISSING CASE mult_versions"
>>>>> ----------------------------------------------^
>>>>> multrec_gen.f90(331): error #5082: Syntax error, found IDENTIFIER 
>>>>> 'STOP' when expecting one of: ( % : . = =>
>>>>>        ERROR STOP "MISSING CASE mult_versions"
>>>>> -------------^
>>>>> multrec_gen.f90(267): error #6404: This name does not have a type, and 
>>>>> must have an explicit type.   [ERROR]
>>>>>      IF (ANY(best_square<1)) ERROR STOP "tiny opts file needs 
>>>>> sufficiently many square sizes"
>>>>> -----------------------------^
>>>>>
>>>>> compilation aborted for multrec_gen.f90 (code 1)
>>>>> More info : 
>>>>> ifort 11.1 
>>>>> ./generate cleanall is done before building with ifort.
>>>>>
>>>>> Thanks for your help.
>>>>>
>>>>> Abhishek 
>>>>>
>>>>> On Wednesday, October 22, 2014 4:31:48 AM UTC-4, Alfio Lazzaro wrote:
>>>>>>
>>>>>> Ok, thanks. I can reproduce your error and I think I understood its 
>>>>>> source. 
>>>>>> It turns out that the compiler used for the compilation of the libsmm 
>>>>>> fortran files (gfortran 4.3 in my case) is the source of the problem.
>>>>>> I think the easiest solution would be to change inside the file:
>>>>>>
>>>>>> build_libsmm/config/linux.intel
>>>>>>
>>>>>> the last line from:
>>>>>>
>>>>>> host_compile="gfortran -O2"
>>>>>>
>>>>>> to
>>>>>>
>>>>>> host_compile="ifort -O2"
>>>>>>
>>>>>> BTW, the error doesn't happen if I use gfortran 4.6.
>>>>>>
>>>>>> I hope this solves your problem.
>>>>>>
>>>>>> Alfio
>>>>>>
>>>>>>
>>>>>> Il giorno mercoledì 22 ottobre 2014 01:44:25 UTC+2, Abhishek 
>>>>>> Bagusetty ha scritto:
>>>>>>>
>>>>>>> Please find the attachment for the tiny summary file. 
>>>>>>>
>>>>>>> Abhishek
>>>>>>>
>>>>>>> On Tuesday, October 21, 2014 11:10:04 AM UTC-4, Alfio Lazzaro wrote:
>>>>>>>>
>>>>>>>> Interesting... Then it seems that the problem is in the small1 
>>>>>>>> phase...
>>>>>>>> Could you send me in attachment your tiny summary file? I will 
>>>>>>>> check what is wrong...
>>>>>>>>
>>>>>>>> Alfio
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Il giorno martedì 21 ottobre 2014 15:24:06 UTC+2, Abhishek 
>>>>>>>> Bagusetty ha scritto:
>>>>>>>>>
>>>>>>>>> I have checked the summary file 
>>>>>>>>> tiny_gen_optimal_dnn_linux.intel.out and it has 9 columns with 13824 lines 
>>>>>>>>> with no sign of corrupted entries. On the same lines, there are 13824 *.out 
>>>>>>>>> files in build_libsmm/run_tiny_dnn/output_linux.intel.
>>>>>>>>>
>>>>>>>>> It looks like we can rule out that there could be something wrong 
>>>>>>>>> with the summary file. I guess it would be worth informing that while 
>>>>>>>>> collecting results using tiny2 some of the *.out files were missing so I 
>>>>>>>>> had to run the tiny1 again to get the completion and tiny2 went smoothly. I 
>>>>>>>>> am not sure if this could have triggered this issue. 
>>>>>>>>>
>>>>>>>>> let me know what you think on this.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Abhishek
>>>>>>>>>
>>>>>>>>> On Tuesday, October 21, 2014 3:20:03 AM UTC-4, Alfio Lazzaro wrote:
>>>>>>>>>>
>>>>>>>>>> OK, the files from tiny1 are generated correctly (6 columns is OK 
>>>>>>>>>> for the tiny1 files). The 9 columns are for the summary file generated 
>>>>>>>>>> during the tiny2 phase in the the main directory. Look inside the main 
>>>>>>>>>> directory (build_libsmm) for the file tiny_gen_optimal_dnn_linux.intel.out. 
>>>>>>>>>> I suspect that this file has 3 columns (the first 3) in some lines (all 
>>>>>>>>>> lines must have 9 columns). BTW, if you hare using the default 
>>>>>>>>>> configuration for tiny1 matrix dimensions, then you should have 13824 *out 
>>>>>>>>>> files inside build_libsmm/run_tiny_dnn/output_linux.intel. You can check it 
>>>>>>>>>> by using
>>>>>>>>>>
>>>>>>>>>> > ls *.out | wc -l
>>>>>>>>>>
>>>>>>>>>> The same for the summary file:
>>>>>>>>>>
>>>>>>>>>> > wc -l  tiny_gen_optimal_dnn_linux.intel.out
>>>>>>>>>> 13824 tiny_gen_optimal_dnn_linux.intel.out
>>>>>>>>>>
>>>>>>>>>> We can have several possible errors:
>>>>>>>>>> 1) some of the *.out files are missing or corrupted
>>>>>>>>>> 2) the summary file is corrupted
>>>>>>>>>>
>>>>>>>>>> Let me know what you get from this investigation. Likely the 
>>>>>>>>>> problem is in the summary file...
>>>>>>>>>>
>>>>>>>>>> Alfio
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Il giorno lunedì 20 ottobre 2014 22:54:39 UTC+2, Abhishek 
>>>>>>>>>> Bagusetty ha scritto:
>>>>>>>>>>>
>>>>>>>>>>> Hi Alfio,
>>>>>>>>>>>
>>>>>>>>>>> I have looked into the *.out files generated by the tiny1. I 
>>>>>>>>>>> didn't see 9 columns in the *.out but instead 6 columns.
>>>>>>>>>>>
>>>>>>>>>>>    1   1   1   1    0.441932       2.263
>>>>>>>>>>>    1   1   1   7    0.242963       4.116
>>>>>>>>>>>    1   1  17   1    0.370944       2.696
>>>>>>>>>>>    1   3   1   1    0.249962       4.001
>>>>>>>>>>>    1   3   1   7    0.544917       1.835
>>>>>>>>>>>    1   5   1   1    0.267959       3.732
>>>>>>>>>>>    1  15   1   1    0.498925       2.004
>>>>>>>>>>>    2   1   1   1    0.441932       2.263
>>>>>>>>>>>
>>>>>>>>>>> This is the output from a file (tiny_find_xx_xx_xx.out) in 
>>>>>>>>>>> /build_libsmm/run_tiny_dnn/output_linux.intel/
>>>>>>>>>>> I have modified the pbs.wlm file by getting rid of few options 
>>>>>>>>>>> like -lmppwidth and -lmppnppn which were not defined in the PBS for our 
>>>>>>>>>>> cluster. 
>>>>>>>>>>> I am not sure how to investigate further. Do you have any 
>>>>>>>>>>> pointers on how to approach this ?
>>>>>>>>>>>
>>>>>>>>>>> Thank for your time. 
>>>>>>>>>>>
>>>>>>>>>>> Abhishek 
>>>>>>>>>>>
>>>>>>>>>>> On Monday, October 20, 2014 12:41:23 AM UTC-4, Alfio Lazzaro 
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hello Abhishek,
>>>>>>>>>>>> could you check if the tiny2 phase is properly producing the 
>>>>>>>>>>>> tiny file? I mean, the error you are seeing is that the system is not able 
>>>>>>>>>>>> to read the tiny results during the small1 phase.
>>>>>>>>>>>> The file should be something similar to:
>>>>>>>>>>>>
>>>>>>>>>>>> 1 1 1    1   1   1   1    0.447362       0.447
>>>>>>>>>>>> 1 1 2    4   1   1   1    0.451188       0.887
>>>>>>>>>>>> 1 1 3    3   1   1   1    0.484057       1.240
>>>>>>>>>>>> 1 1 4    5   1   1   4    0.500210       1.599
>>>>>>>>>>>> 1 1 5    6   1   1   5    0.548113       1.824
>>>>>>>>>>>> 1 1 6    1   1   1   6    0.456724       2.190
>>>>>>>>>>>>
>>>>>>>>>>>> As you can see there are 9 columns. If you see only 3 (the 
>>>>>>>>>>>> first three ones) then there is something wrong during tiny1.
>>>>>>>>>>>> The next step would be to investigate if the files *.out inside 
>>>>>>>>>>>> the tiny1 directory and then output directory are correct (you can post an 
>>>>>>>>>>>> example of .out file).
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Alfio
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Il giorno domenica 19 ottobre 2014 22:03:42 UTC+2, Abhishek 
>>>>>>>>>>>> Bagusetty ha scritto:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Users & Developers,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am trying to build libsmm small matrix multiplication 
>>>>>>>>>>>>> library and got an error while doing the 3rd step.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Steps : 
>>>>>>>>>>>>> 1. ./generate -c config/linux.intel -j 100 -t 16 -w pbs tiny1
>>>>>>>>>>>>> 2. ./generate -c config/linux.intel tiny2
>>>>>>>>>>>>> 3. ./generate -c config/linux.intel -j 20 -t 16 -w pbs small1
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> multrec_gen.f90:267.29:
>>>>>>>>>>>>>
>>>>>>>>>>>>>      IF (ANY(best_square<1)) ERROR STOP "tiny opts file needs 
>>>>>>>>>>>>> sufficiently many
>>>>>>>>>>>>>
>>>>>>>>>>>>>                              1
>>>>>>>>>>>>>
>>>>>>>>>>>>> Error: Cannot assign to a named constant at (1)
>>>>>>>>>>>>>
>>>>>>>>>>>>> multrec_gen.f90:331.7:
>>>>>>>>>>>>>
>>>>>>>>>>>>>        ERROR STOP "MISSING CASE mult_versions"
>>>>>>>>>>>>>
>>>>>>>>>>>>>        1
>>>>>>>>>>>>> Error: Unclassifiable statement at (1)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> It would be great, if anyone has any pointers to get a fix on 
>>>>>>>>>>>>> this issue. 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>>>> -----------------------------------------------
>>>>>>>>>>>>> Abhishek Bagusetty
>>>>>>>>>>>>> PhD Student, Computational Modeling & Simulation
>>>>>>>>>>>>> Center for Simulation and Modeling
>>>>>>>>>>>>> Department of Chemical & Petroleum Engineering
>>>>>>>>>>>>> University of Pittsburgh
>>>>>>>>>>>>> Pittsburgh, PA - 15261
>>>>>>>>>>>>> Office : 920 Benedum Hall
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----------------------------------------------------------------------------------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20141030/fdd1efe5/attachment.htm>