Can't read Graphml files on Linux Mint 18 (works on mint 17.1)

[ntamas]

From @swayson on September 28, 2016 14:45

Hi there,

I have had issues reading graphml using python-igraph 7.1 via pip on Mint 18. I however, don't experience this issue on Mint 17.1.

Note I did install libxml2-dev in my attempts.

I get the following error on python 3.5

ig.read('test.graphml')

/home/username/miniconda3/lib/python3.5/site-packages/igraph/__init__.py in read(filename, *args, **kwds)
   4061     @param filename: the name of the file to be loaded
   4062     """
-> 4063     return Graph.Read(filename, *args, **kwds)
   4064 load=read
   4065 

/home/username/miniconda3/lib/python3.5/site-packages/igraph/__init__.py in Read(klass, f, format, *args, **kwds)
   2221             raise IOError("no reader method for file format: %s" % str(format))
   2222         reader = getattr(klass, reader)
-> 2223         return reader(f, *args, **kwds)
   2224     Load = Read
   2225 

SystemError: <built-in method Read_GraphML of type object at 0x14c2758> returned NULL without setting an error

On Python 2.7 I had different errors when running the same command.

NotImplementedError: Error at foreign-graphml.c:1211: GraphML support is disabled, Unimplemented function call

Any ideas?

Copied from original issue: igraph/igraph#965

[ntamas]

@swayson I'll need to know more about your exact setup:

• How did you install the C core of igraph? Is there a package for the C core of igraph in Linux Mint 18, or did you compile the C core yourself?
• If you compiled the C core yourself, did you compile it before or after installing libxml2-dev?
• If you did not compile the C core yourself, did you let pip install python-igraph download and compile the source code of the C core on its own? If so, was libxml2-dev installed when you invoked pip install python-igraph?
• Do you have pkg-config on your system? What does pkg-config --cflags --libs igraph print?

[swayson]

Thank you for the quick response :)

Initially we installed igraph on mint18 using pip and at that stage libxml2-dev was not installed. This was using python 3.5 and we used a conda environment to keep things isolated. We then thought, hmm, maybe this relates to python 3.5 so we then tried python 2.7 and were unsuccessful. Later on did a bit of searching and noticed people with this error often reference libxml2-dev, so installed that and tried this process again. It seem to have not made a difference. As a final resort we tried $ sudo apt-get install python-igraph but that too did not make a difference.

Then I moved this to the mint 17.1 VM and followed the same process. So libxml2-dev was not installed and pip failed. We then installed libxml2-dev, and pip worked. Then tried reading the graphml and it worked. This was the case for both python 2.7 and python 3.5.

Hope this all makes sense and is clear. It was quite a trial and error process.

Regarding your last point, I get this for both mint 18 and mint 17

$ pkg-config --cflags --libs igraph
Package igraph was not found in the pkg-config search path.
Perhaps you should add the directory containing `igraph.pc'
to the PKG_CONFIG_PATH environment variable
No package 'igraph' found

[ntamas]

Okay, so this is pretty complicated. As you surely know, igraph itself is a library written in C and python-igraph is just an interface on top of it that allows you to use igraph from Python. The functionality offered by python-igraph depends a lot on the underlying C library; for instance, if the C core was compiled without XML support, then you won't be able to read GraphML files from Python. This is what happened in your case.

There are basically four options to install the Python interface of igraph, in increasing order of complexity:

1. If your OS offers precompiled packages for igraph and python-igraph, then most likely this is what you need because the precompiled packages are usually built with XML support. In this case, you need to remove everything that you have compiled or installed manually, and just use the package manager to install the packages for the C core of igraph and its Python interface. (In Ubuntu and Debian, these are called libigraph0 and python-igraph, respectively). However, this will install igraph only for a specific version and installation of Python (the one that comes with the OS by default), so if you are using Anaconda Python or any other third-party Python distribution, then this option won't work for you.
2. You can also install only the C core of igraph from the package manager (i.e. libigraph0) and then compile the Python interface yourself. If you go this way, you'll also need the development libraries for Python and the C core of igraph; these are usually called libigraph0-dev and python-dev, respectively. Having installed these, you can then run pip install python-igraph, which will download and install the Python interface from PyPI. The catch here is that pip would simply download the latest version from PyPI (that is, 0.7.1-post6), which is suitable for igraph 0.7.1 only. So, if your OS provides an older version of libigraph0 and not 0.7.1, you need to download the corresponding Python interface from PyPI manually, extract it and then run python setup.py install. The Python interface relies on pkg-config to find where the C core is installed, so you'll also need pkg-config (and of course libigraph0-dev, because this contains the configuration file for pkg-config that tells where to find the C core of igraph).
3. You can compile both the C core and the Python interface on your own. The source code of the C core can be obtained from http://igraph.org/c/ - you'll need to extract it and then run the usual ./configure && make && sudo make install dance. At the end of the ./configure phase, there is a short summary of the configuration of the library - this tells you whether it will be compiled with GraphML support or not. If the output of ./configure tells that you won't have GraphML support, it means that pkg-config failed to find libxml2 on your system. Once the C core is installed, you can install the Python interface the same way you would have done with option 2 above.
4. You can also run pip install python-igraph without installing the C core. When the setup script for the Python interface is executed, it tries to find the C core first using pkg-config, and when it is unable to do so, it will attempt to download and compile the C core on its own. Otherwise it is essentially the same as step 3. However, the installation won't pause after the ./configure step so it is harder to check whether libxml2 was picked up properly by the configuration script.

At this stage, I would probably recommend you to start with a clean slate (i.e. remove everything igraph-related from your machine), install libxml2-dev and pkg-config from the package manager, and then run pip install python-igraph. Hopefully it will do the right thing and find libxml2 properly. If it did not, please get back to me and then we'll iterate further.

[imsodin]

I encounter the same error message with Graph.Read_Lgl. In installed both the C core and Python interface from debian repos, pypi and source and the error is the same all the time. The configure stage did not show any obvious problems - there are some "no"s, but I they don't seem related: https://gist.github.com/imsodin/6dc4d8694ed4fefea7fd3e80feba777d

Any tips how to investigate this further?

[ntamas]

configure log seems okay. Are you using Python 2.x or 3.x? What is the exact error message? (The original post contains two, one for Python 2.x and one for 3.x)

[imsodin]

Wow, that was fast :)

The error is

SystemError                               Traceback (most recent call last)
/media/ntfs_data/Dokumente/ETH/complex_networks/ex01/EX01.py in <module>()
     25     task2_1()
     26     task2_2()
---> 27     task3_1()

/media/ntfs_data/Dokumente/ETH/complex_networks/ex01/EX01.py in task3_1()
     18 
     19 def task3_1():
---> 20     g = ig.Graph.Read_Lgl("./EX01_gentoo_810_30d.txt")
     21     print(g)
     22 

SystemError: <built-in method Read_Lgl of type object at 0x144c988> returned NULL without setting an error

and python version

python -V: Python 3.6.4+

from debian repos.

[ntamas]

Okay, so this is probably specific to Python 3.x. I will test it in a virtualenv tomorrow and get back to you.

[ntamas]

Managed to reproduce this on Mac OS X with Python 3.6.3. I will post a patch soon.

Update: it seems like the issue happens only when you are trying to read a file that is not an LGL file. Are you sure that the input file is an LGL file and not, say, a plain edgelist file?

[ntamas]

FIxed in f7f612c; for the record, the problem was that in cases when an error happened while loading a graph (typically a parse error), the cleanup process that released the file handle has accidentally cleared the exception flag in Python 3.x, so the reading function (e.g., Read_LGL) returned with NULL but without an approriate exception. This has happened only if there was an error while reading the input file. @imsodin, please check whether the format of your input file is correct (i.e. whether it is really an LGL file), otherwise the most likely explanation is that igraph_read_graph_lgl returned an error code in the C layer, which has gotten translated into a Python exception that was then later swallowed by the buggy Python interface -- that's why you have seen a SystemError instead.

[imsodin]

Nice, applied to the latest release that returns a parse error and you are right, it seems to be a an edgelist indeed. However both Read_Edgelist or just Read (which identifies it as an edgelist) also return a generic parse error. So this is now either a support request or an igraph bug, however it would still be highly appreciated if you could have a quick look at the file causing this error, maybe the cause is obvious (and potentially showing a lot of ignorance on my side :P ):
https://gist.github.com/imsodin/dc0b55dda23ce3d167c526d1385933dc

[ntamas]

Edgelist files should contain numeric edge IDs from 0 to |V|-1 only. What you have is a "named edgelist", which can be read with Graph.Read_Ncol instead. (Graph.Read identifies the file simply based on its extension, and well, .txt is a pretty generic one so it just assumes it's an edgelist).