Difficulties uploading datasets

[joaquinvanschoren]

When testing the dataset uploads, an outdated URL is used

dataset = OpenMLDataset(
name="UploadTestWithURL", version=1, description="test",
format="ARFF",
url="http://expdb.cs.kuleuven.be/expdb/data/uci/nominal/iris.arff")
dataset.publish()

Should be https://www.openml.org/data/download/61/dataset_61_iris.arff

[joaquinvanschoren]

Weirdly, this also fails. The test server fails to parse the ARFF file?

dataset = oml.datasets.OpenMLDataset(
            name="UploadTestWithURL", version=1, description="test",
            format="ARFF",
            url="https://www.openml.org/data/download/61/dataset_61_iris.arff")
dataset.publish()

---------------------------------------------------------------------------
OpenMLServerException                     Traceback (most recent call last)
<ipython-input-58-99a1903d7914> in <module>()
      9             format="ARFF",
     10             url="https://www.openml.org/data/download/61/dataset_61_iris.arff")
---> 11 dataset.publish()
     12 
     13 

/Users/joa/anaconda/lib/python3.5/site-packages/openml/datasets/dataset.py in publish(self)
    392 
    393         return_value = _perform_api_call("/data/", file_dictionary=file_dictionary,
--> 394                                          file_elements=file_elements)
    395 
    396         self.dataset_id = int(xmltodict.parse(return_value)['oml:upload_data_set']['oml:id'])

/Users/joa/anaconda/lib/python3.5/site-packages/openml/_api_calls.py in _perform_api_call(call, data, file_dictionary, file_elements, add_authentication)
     49     if file_dictionary is not None or file_elements is not None:
     50         return _read_url_files(url, data=data, file_dictionary=file_dictionary,
---> 51                                file_elements=file_elements)
     52     return _read_url(url, data)
     53 

/Users/joa/anaconda/lib/python3.5/site-packages/openml/_api_calls.py in _read_url_files(url, data, file_dictionary, file_elements)
     95     response = requests.post(url, data=data, files=file_elements)
     96     if response.status_code != 200:
---> 97         raise _parse_server_exception(response)
     98     if 'Content-Encoding' not in response.headers or \
     99             response.headers['Content-Encoding'] != 'gzip':

OpenMLServerException: Error parsing dataset ARFF file

[joaquinvanschoren]

Sorry, 3rd question on this topic: how can I upload a dataset that is locally stored?
I tried stating the file path with 'data_file', and while an OpenMLDataset object is created, during upload in fails with the same parsing error as above.

[janvanrijn]

Answers to the questions:
a) This is done on purpose. We wanted an ARFF from the web, and this one seemed appropriate. Connection between openml and expdb is coincidental.
b) I would guess that the arff of iris (and many other UCI arffs) do not comply with the arff standards required by the openml server (it mixes tabs and spaces)
c) cannot answer this ATM

[joaquinvanschoren]

a) I meant that that kuleuven server is no longer available. The URL gives a 404.
b) Fair enough. It is really hard to see this though :)
c) Works now.

[joaquinvanschoren]

All good, except for that url in the unit test.

[mfeurer]

Hm, why don't the unit tests fail then? https://travis-ci.org/openml/openml-python/builds/264404343

[joaquinvanschoren]

The error message has changed, but this still looks a bug.

dataset = openml.datasets.OpenMLDataset(
name="iristest", version=1, description="test",
format="ARFF",
url="http://expdb.cs.kuleuven.be/expdb/data/uci/nominal/iris.arff")
dataset.publish()

---------------------------------------------------------------------------
ExpatError                                Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/openml/_api_calls.py in _parse_server_exception(response, url)
    130     try:
--> 131         server_exception = xmltodict.parse(response.text)
    132     except:

~/anaconda3/lib/python3.6/site-packages/xmltodict.py in parse(xml_input, encoding, expat, process_namespaces, namespace_separator, disable_entities, **kwargs)
    329     else:
--> 330         parser.Parse(xml_input, True)
    331     return handler.item

ExpatError: no element found: line 1, column 0

During handling of the above exception, another exception occurred:

OpenMLServerError                         Traceback (most recent call last)
<ipython-input-20-fa1103f9ad92> in <module>()
----> 1 dataset.publish()

~/anaconda3/lib/python3.6/site-packages/openml/datasets/dataset.py in publish(self)
    423 
    424         return_value = _perform_api_call("/data/", file_dictionary=file_dictionary,
--> 425                                          file_elements=file_elements)
    426 
    427         self.dataset_id = int(xmltodict.parse(return_value)['oml:upload_data_set']['oml:id'])

~/anaconda3/lib/python3.6/site-packages/openml/_api_calls.py in _perform_api_call(call, data, file_dictionary, file_elements, add_authentication)
     50     if file_dictionary is not None or file_elements is not None:
     51         return _read_url_files(url, data=data, file_dictionary=file_dictionary,
---> 52                                file_elements=file_elements)
     53     return _read_url(url, data)
     54 

~/anaconda3/lib/python3.6/site-packages/openml/_api_calls.py in _read_url_files(url, data, file_dictionary, file_elements)
     96     response = requests.post(url, data=data, files=file_elements)
     97     if response.status_code != 200:
---> 98         raise _parse_server_exception(response, url=url)
     99     if 'Content-Encoding' not in response.headers or \
    100             response.headers['Content-Encoding'] != 'gzip':

~/anaconda3/lib/python3.6/site-packages/openml/_api_calls.py in _parse_server_exception(response, url)
    133         raise OpenMLServerError(('Unexpected server error. Please '
    134                                  'contact the developers!\nStatus code: '
--> 135                                  '%d\n' % response.status_code) + response.text)
    136 
    137     code = int(server_exception['oml:error']['oml:code'])

OpenMLServerError: Unexpected server error. Please contact the developers!
Status code: 500

[mfeurer]

@joaquinvanschoren did you try the latest dataset upload? Is it the way you expect it? If yes, can we close this?

[joaquinvanschoren]

I'll try to test it over the weekend

On Fri, 8 Jun 2018 at 13:41 Matthias Feurer ***@***.***> wrote: @joaquinvanschoren <https://github.com/joaquinvanschoren> did you try the latest dataset upload? Is it the way you expect it? If yes, can we close this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#292 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQV7TdPSmN4rpWUsAzfWCELw1E4Gv-ks5t6mLbgaJpZM4O11E1> .

-- Thank you, Joaquin

[mfeurer]

@joaquinvanschoren did you recently try uploading a dataset? Can this be closed?

[mfeurer]

Closing this due to inactivity. If this is still an issue with the recent 0.8 release, please reopen.