<div dir="ltr">Hi Steve, I did not realize that you had such an elaborate library project going on, pretty cool :-) I added your project our collections of tools: http://cp2k.org/tools:pwtools Feel free to elaborate on the page. > The numpy.fromfile() idea didn't actually occur to me even though it is pretty obvious, thanks. You're welcome. I came up with that trick when I wrote a parser for binary gromacs trajectories, a couple years ago. > I also adapted an earlier Fortran + f2py version to cp2k files. It is > much faster than the simple Python version, even though it walks the > file twice (once to find nstep, rather brute-force, might change later). Hmm, looks like your are still doing quite a bit of processing in python. In principle one should be able to get pretty close to the Fortran performance by just using numpy routines. But I guess you are aware of that ;-) -Ole Am Montag, 1. Juni 2015 20:35:14 UTC+2 schrieb Steve Schmerler:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;">On May 22 01:54 -0700, Ole Schütt wrote: > Yes, exactly. Maybe as <a href="http://cp2k.org/tools:dumpdcd" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcp2k.org%2Ftools%3Adumpdcd\46sa\75D\46sntz\0751\46usg\75AFQjCNGViVPLLE2EeZHZTphOWzU019E5Rg';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcp2k.org%2Ftools%3Adumpdcd\46sa\75D\46sntz\0751\46usg\75AFQjCNGViVPLLE2EeZHZTphOWzU019E5Rg';return true;">http://cp2k.org/tools:dumpdcd</a> or > <a href="http://cp2k.org/howto:read_dcd_files" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcp2k.org%2Fhowto%3Aread_dcd_files\46sa\75D\46sntz\0751\46usg\75AFQjCNH7WOVV8t9kfgu3AfBqMprEiPPqLw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcp2k.org%2Fhowto%3Aread_dcd_files\46sa\75D\46sntz\0751\46usg\75AFQjCNH7WOVV8t9kfgu3AfBqMprEiPPqLw';return true;">http://cp2k.org/howto:read_dcd_files</a> . > > For what solution did you settle now? Are you using a combination of seek() > and numpy.fromfile() to parse the DCD file directly, or are you calling > dumpdcd in a pipe? The numpy.fromfile() idea didn't actually occur to me even though it is pretty obvious, thanks. Based on this I wrote a simple numpy-only version. It uses a rather simple approach, is not super-fast and can certainly be optimized. It took me a while to read the header correctly since it has "marker" integers (84, 164, ...) around each block of data, which are the size (in bytes) of the block they enclose. In Fortran, one can apparently just read the data without reading the markers explicitly. Again, VMDs dcdplugin.c saved the day. The dcd reader is part of my "pwtools" project [1], but the dcd part is (almost) independent from the rest - you only need the file "dcp.py" [2]. The Python version is called "read_dcd_data_py": $ python >>> import dcd >>> cc,co = dcd.read_dcd_data_py('cp2k.dcd') where "cc" is a (nstep,6) array (float64) of unit cell parameters (a,b,c,alpha,beta,gamma) -- make sure you use dcd_aligned_cell in cp2k. "co" is a (nstep,natoms,3) array (float32) of cartesian coordinates in Angstrom. Have a look at the doc string for how to read lammps-style files (keyword convang=True). I also adapted an earlier Fortran + f2py version to cp2k files. It is much faster than the simple Python version, even though it walks the file twice (once to find nstep, rather brute-force, might change later). To use this stand-alone, copy dcd.f90 somewhere and run f2py (part of numpy) to compile the extension module into "_dcd.so", which you import with "import _dcd": $ f2py -c dcd.f90 -m _dcd $ python >>> import _dcd >>> nstep, natoms, timestep = _dcd.get_dcd_file_info('cp2k.dcd', nstephdr=False) >>> cc,co = _dcd.read_dcd_data('cp2k.dcd', nstep, natoms, convang=False) There is also a wrapper function in dcd.py called "read_dcd_data_f", which does exactly this. Again, please have a look at the doc string for more examples. I don't use this regularly on large files at the moment, so there may be issues that I'm not aware of. If you find bugs, have better / faster versions etc, please let me know. best, Steve [1] <a href="https://bitbucket.org/elcorto/pwtools" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools\46sa\75D\46sntz\0751\46usg\75AFQjCNGmhdEKjkN_hF8GntFzJ2bDTUYLbg';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools\46sa\75D\46sntz\0751\46usg\75AFQjCNGmhdEKjkN_hF8GntFzJ2bDTUYLbg';return true;">https://bitbucket.org/elcorto/pwtools</a> Get it: hg clone <a href="https://bitbucket.org/elcorto/pwtools" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools\46sa\75D\46sntz\0751\46usg\75AFQjCNGmhdEKjkN_hF8GntFzJ2bDTUYLbg';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools\46sa\75D\46sntz\0751\46usg\75AFQjCNGmhdEKjkN_hF8GntFzJ2bDTUYLbg';return true;">https://bitbucket.org/elcorto/pwtools</a> or wget <a href="https://bitbucket.org/elcorto/pwtools/get/e591501fa367.zip" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools%2Fget%2Fe591501fa367.zip\46sa\75D\46sntz\0751\46usg\75AFQjCNGdGgnFN2lzvcbx3P1c87YN0k7s6w';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools%2Fget%2Fe591501fa367.zip\46sa\75D\46sntz\0751\46usg\75AFQjCNGdGgnFN2lzvcbx3P1c87YN0k7s6w';return true;">https://bitbucket.org/elcorto/pwtools/get/e591501fa367.zip</a> [2] <a href="https://bitbucket.org/elcorto/pwtools/src/e591501fa3671cab7f7583ad747541cb14512d55/dcd.py?at=default" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools%2Fsrc%2Fe591501fa3671cab7f7583ad747541cb14512d55%2Fdcd.py%3Fat%3Ddefault\46sa\75D\46sntz\0751\46usg\75AFQjCNEYe6b1un2zfNsJijwOrofdGXlhDw';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fbitbucket.org%2Felcorto%2Fpwtools%2Fsrc%2Fe591501fa3671cab7f7583ad747541cb14512d55%2Fdcd.py%3Fat%3Ddefault\46sa\75D\46sntz\0751\46usg\75AFQjCNEYe6b1un2zfNsJijwOrofdGXlhDw';return true;">https://bitbucket.org/elcorto/pwtools/src/e591501fa3671cab7f7583ad747541cb14512d55/dcd.py?at=default</a> -- Steve Schmerler Institut für Theoretische Physik TU Freiberg, Germany </blockquote></div>