Task creation not supported?

[joaquinvanschoren]

I tried to create a task via the Python API like this:

task = oml.tasks.OpenMLTask(
            task_type = 'Supervised Classification', 
            data_set_id = 602,
            target_name = 'class', 
            estimation_procedure_type = 'crossvalidation', 
            estimation_parameters = {'number_folds': '10',
                                     'number_repeats': '1',
                                     'percentage': '',
                                     'stratified_sampling': 'true'})

But I'm getting:

TypeError: __init__() missing 5 required positional arguments: 'task_id', 'task_type_id', 'data_splits_url', 'evaluation_measure', and 'cost_matrix'

None of these make sense for creating a new simple task with cross-validation. Why are they require. Setting them to None also doesn't help, since 'task_id' needs to be a number.

Am I missing something?

[mfeurer]

Local task creation is not implemented. Therefore, all required attributes need to be specified. If we wanted to implement this, I think we'd need to go a different way, though. Something like create_task_on_OpenML, which is aware of all different kinds of parameters that can be passed.

[joaquinvanschoren]

Sure, but if I write create_task_on_OpenML, I want to do that based on the OpenMLTask object, no? Right now I cannot make a task object if the task_id isn't already known.

How do I proceed?

[mfeurer]

I'm not sure what you want to achieve.

[joaquinvanschoren]

Very simple: say I have just uploaded a task using the Python API. Now I want to create a classification task on it. I don't want to do this via the website, e.g. because I want to create many tasks, or I have a custom holdout set and putting in all those numbers in the web form is annoying. Also, the Java API allows task creation, why not the Python API?

On Tue, Aug 15, 2017 at 2:49 PM Matthias Feurer ***@***.***> wrote: I'm not sure what you want to achieve. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQV8_N-wpkvFh8lUQ6wq-nZ-dxsb7Kks5sYQcegaJpZM4O2-yH> .

-- Thank you, Joaquin

[joaquinvanschoren]

Maybe I misunderstood. Are you saying I should just call the REST API without creating a task object first? That could work. On Tue, Aug 15, 2017 at 2:51 PM Joaquin Vanschoren < joaquin.vanschoren@gmail.com> wrote:

Very simple: say I have just uploaded a task using the Python API. Now I want to create a classification task on it. I don't want to do this via the website, e.g. because I want to create many tasks, or I have a custom holdout set and putting in all those numbers in the web form is annoying. Also, the Java API allows task creation, why not the Python API? On Tue, Aug 15, 2017 at 2:49 PM Matthias Feurer ***@***.***> wrote: > I'm not sure what you want to achieve. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#294 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABpQV8_N-wpkvFh8lUQ6wq-nZ-dxsb7Kks5sYQcegaJpZM4O2-yH> > . > -- Thank you, Joaquin

-- Thank you, Joaquin

[mfeurer]

Okay, your use case makes sense. The reason why this isn't supported in the python API is because no one needed this feature so far.

And yes, using the REST API without creating a task object is basically what I wanted to say.

[joaquinvanschoren]

OK, but just to be sure: when we upload a dataset, we first create a dataset object, and then export that to xml for the description, and send that to the REST API. Do we want to do this differently for task uploads? Maybe I should create a task object anyway with dummy values (e.g. task_id 0) since this will be ignored by the server on upload anyway? Seems easier? ``` def publish(self): """Publish the dataset on the OpenML server. Upload the dataset description and dataset content to openml. Returns ------- return_code : int Return code from server return_value : string xml return from server """ file_elements = {'description': self._to_xml()} file_dictionary = {} if self.data_file is not None: file_dictionary['dataset'] = self.data_file return_value = _perform_api_call("/data/", file_dictionary=file_dictionary, file_elements=file_elements) self.dataset_id = int(xmltodict.parse(return_value)['oml:upload_data_set']['oml:id']) return self ```

On Tue, Aug 15, 2017 at 2:56 PM Matthias Feurer ***@***.***> wrote: Okay, your use case makes sense. The reason why this isn't supported in the python API is because no one needed this feature so far. And yes, using the REST API without creating a task object is basically what I wanted to say. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQV5Zn_yFf0IvGxmmYvSjAWxwsCne7ks5sYQjsgaJpZM4O2-yH> .

-- Thank you, Joaquin

[mfeurer]

Yep, you're right. My proposal would make the API cluttered and hard to understand. Creating a task object to upload would be way easier and consistent with the rest of the API. Thanks for pointing that out.

[joaquinvanschoren]

OK, I'm giving it a go. The task upload XSD is really weird, but it will do for now.

On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer ***@***.***> wrote: Yep, you're right. My proposal would make the API cluttered and hard to understand. Creating a task object to upload would be way easier and consistent with the rest of the API. Thanks for pointing that out. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> .

-- Thank you, Joaquin

[joaquinvanschoren]

2 things: 1) when I download a task, I get the same info twice: ``` 'estimation_parameters': {'number_folds': '10', 'number_repeats': '1', 'percentage': '', 'stratified_sampling': 'true'}, 'estimation_procedure': {'data_splits_url': 'https://www.openml.org/api_splits/get/59/Task_59_splits.arff', 'parameters': {'number_folds': '10', 'number_repeats': '1', 'percentage': '', 'stratified_sampling': 'true'}, 'type': 'crossvalidation'}, ``` Shall I just remove the first 'estimation_parameters'? The API doesn't return it like this, only the python API does. 2) when creating a task, the user needs to be able to state the estimation procedure. This is now done by giving the task type id and estimation procedure id, for instance 10 fold CV for classification has task type 1 and estimation procedure 1. This is a bit hard to look up. Shall I allow people to state something like task_type='Supervised Classification' and estimation_procedure='10-fold Cross-validation'? I can do an easy lookup in the API, and it is a whole lot easier. What do you think? On Tue, Aug 15, 2017 at 3:57 PM Joaquin Vanschoren < joaquin.vanschoren@gmail.com> wrote:

OK, I'm giving it a go. The task upload XSD is really weird, but it will do for now. On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer ***@***.***> wrote: > Yep, you're right. My proposal would make the API cluttered and hard to > understand. Creating a task object to upload would be way easier and > consistent with the rest of the API. Thanks for pointing that out. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#294 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> > . > -- Thank you, Joaquin

-- Thank you, Joaquin

[janvanrijn]

Ideally, this lookup happens on the client side. 2017-08-16 7:27 GMT+02:00 Joaquin Vanschoren <notifications@github.com>:

…

2 things: 1) when I download a task, I get the same info twice: ``` 'estimation_parameters': {'number_folds': '10', 'number_repeats': '1', 'percentage': '', 'stratified_sampling': 'true'}, 'estimation_procedure': {'data_splits_url': 'https://www.openml.org/api_splits/get/59/Task_59_splits.arff', 'parameters': {'number_folds': '10', 'number_repeats': '1', 'percentage': '', 'stratified_sampling': 'true'}, 'type': 'crossvalidation'}, ``` Shall I just remove the first 'estimation_parameters'? The API doesn't return it like this, only the python API does. 2) when creating a task, the user needs to be able to state the estimation procedure. This is now done by giving the task type id and estimation procedure id, for instance 10 fold CV for classification has task type 1 and estimation procedure 1. This is a bit hard to look up. Shall I allow people to state something like task_type='Supervised Classification' and estimation_procedure='10-fold Cross-validation'? I can do an easy lookup in the API, and it is a whole lot easier. What do you think? On Tue, Aug 15, 2017 at 3:57 PM Joaquin Vanschoren < ***@***.***> wrote: > OK, I'm giving it a go. The task upload XSD is really weird, but it will > do for now. > > On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer < ***@***.***> > wrote: > >> Yep, you're right. My proposal would make the API cluttered and hard to >> understand. Creating a task object to upload would be way easier and >> consistent with the rest of the API. Thanks for pointing that out. >> >> — >> You are receiving this because you authored the thread. >> Reply to this email directly, view it on GitHub >> <#294# issuecomment-322370079>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_ owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> >> . >> > -- > Thank you, > Joaquin > -- Thank you, Joaquin — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACL7-iUiBnY_dKAe2UNOsWBI8jl6IQFFks5sYn3HgaJpZM4O2-yH> .

[joaquinvanschoren]

I can't look it up on the client side, unless there is a new API call for this?

On Wed, 16 Aug 2017 at 21:19, janvanrijn ***@***.***> wrote: Ideally, this lookup happens on the client side. 2017-08-16 7:27 GMT+02:00 Joaquin Vanschoren ***@***.***>: > 2 things: > > 1) when I download a task, I get the same info twice: > ``` > > 'estimation_parameters': {'number_folds': '10', > 'number_repeats': '1', > 'percentage': '', > 'stratified_sampling': 'true'}, > 'estimation_procedure': {'data_splits_url': > 'https://www.openml.org/api_splits/get/59/Task_59_splits.arff', > 'parameters': {'number_folds': '10', > 'number_repeats': '1', > 'percentage': '', > 'stratified_sampling': 'true'}, > 'type': 'crossvalidation'}, > > ``` > > > Shall I just remove the first 'estimation_parameters'? The API doesn't > return it like this, only the python API does. > > > 2) when creating a task, the user needs to be able to state the estimation > procedure. This is now done by giving the task type id and estimation > procedure id, for instance 10 fold CV for classification has task type 1 > and estimation procedure 1. > This is a bit hard to look up. Shall I allow people to state something like > task_type='Supervised Classification' and estimation_procedure='10-fold > Cross-validation'? I can do an easy lookup in the API, and it is a whole > lot easier. > > What do you think? > > > > On Tue, Aug 15, 2017 at 3:57 PM Joaquin Vanschoren < > ***@***.***> wrote: > > > OK, I'm giving it a go. The task upload XSD is really weird, but it will > > do for now. > > > > On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer < > ***@***.***> > > wrote: > > > >> Yep, you're right. My proposal would make the API cluttered and hard to > >> understand. Creating a task object to upload would be way easier and > >> consistent with the rest of the API. Thanks for pointing that out. > >> > >> — > >> You are receiving this because you authored the thread. > >> Reply to this email directly, view it on GitHub > >> <#294# > issuecomment-322370079>, > >> or mute the thread > >> < https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_ > owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> > >> . > >> > > -- > > Thank you, > > Joaquin > > > -- > Thank you, > Joaquin > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #294 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ACL7-iUiBnY_dKAe2UNOsWBI8jl6IQFFks5sYn3HgaJpZM4O2-yH > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQVyrtumeMw9Z3vONcxZPHSWAzbQGHks5sYrQQgaJpZM4O2-yH> .

-- Thank you, Joaquin

[janvanrijn]

https://www.openml.org/api/v1/estimationprocedure/list 2017-08-16 11:22 GMT+02:00 Joaquin Vanschoren <notifications@github.com>:

…

I can't look it up on the client side, unless there is a new API call for this? On Wed, 16 Aug 2017 at 21:19, janvanrijn ***@***.***> wrote: > Ideally, this lookup happens on the client side. > > 2017-08-16 7:27 GMT+02:00 Joaquin Vanschoren ***@***.***>: > > > 2 things: > > > > 1) when I download a task, I get the same info twice: > > ``` > > > > 'estimation_parameters': {'number_folds': '10', > > 'number_repeats': '1', > > 'percentage': '', > > 'stratified_sampling': 'true'}, > > 'estimation_procedure': {'data_splits_url': > > 'https://www.openml.org/api_splits/get/59/Task_59_splits.arff', > > 'parameters': {'number_folds': '10', > > 'number_repeats': '1', > > 'percentage': '', > > 'stratified_sampling': 'true'}, > > 'type': 'crossvalidation'}, > > > > ``` > > > > > > Shall I just remove the first 'estimation_parameters'? The API doesn't > > return it like this, only the python API does. > > > > > > 2) when creating a task, the user needs to be able to state the > estimation > > procedure. This is now done by giving the task type id and estimation > > procedure id, for instance 10 fold CV for classification has task type 1 > > and estimation procedure 1. > > This is a bit hard to look up. Shall I allow people to state something > like > > task_type='Supervised Classification' and estimation_procedure='10-fold > > Cross-validation'? I can do an easy lookup in the API, and it is a whole > > lot easier. > > > > What do you think? > > > > > > > > On Tue, Aug 15, 2017 at 3:57 PM Joaquin Vanschoren < > > ***@***.***> wrote: > > > > > OK, I'm giving it a go. The task upload XSD is really weird, but it > will > > > do for now. > > > > > > On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer < > > ***@***.***> > > > wrote: > > > > > >> Yep, you're right. My proposal would make the API cluttered and hard > to > > >> understand. Creating a task object to upload would be way easier and > > >> consistent with the rest of the API. Thanks for pointing that out. > > >> > > >> — > > >> You are receiving this because you authored the thread. > > >> Reply to this email directly, view it on GitHub > > >> <#294# > > issuecomment-322370079>, > > >> or mute the thread > > >> < > https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_ > > owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> > > >> . > > >> > > > -- > > > Thank you, > > > Joaquin > > > > > -- > > Thank you, > > Joaquin > > > > — > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub > > < > #294# issuecomment-322669881 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/ACL7-iUiBnY_ dKAe2UNOsWBI8jl6IQFFks5sYn3HgaJpZM4O2-yH > > > > . > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#294# issuecomment-322714244>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ ABpQVyrtumeMw9Z3vONcxZPHSWAzbQGHks5sYrQQgaJpZM4O2-yH> > . > -- Thank you, Joaquin — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACL7-t7H65ojsqFIYwiePjW-EgUR2mvoks5sYrTxgaJpZM4O2-yH> .

[joaquinvanschoren]

Ah cool, will document that too :)

On Wed, 16 Aug 2017 at 21:49, janvanrijn ***@***.***> wrote: https://www.openml.org/api/v1/estimationprocedure/list 2017-08-16 11:22 GMT+02:00 Joaquin Vanschoren ***@***.***>: > I can't look it up on the client side, unless there is a new API call for > this? > On Wed, 16 Aug 2017 at 21:19, janvanrijn ***@***.***> wrote: > > > Ideally, this lookup happens on the client side. > > > > 2017-08-16 7:27 GMT+02:00 Joaquin Vanschoren ***@***.*** >: > > > > > 2 things: > > > > > > 1) when I download a task, I get the same info twice: > > > ``` > > > > > > 'estimation_parameters': {'number_folds': '10', > > > 'number_repeats': '1', > > > 'percentage': '', > > > 'stratified_sampling': 'true'}, > > > 'estimation_procedure': {'data_splits_url': > > > 'https://www.openml.org/api_splits/get/59/Task_59_splits.arff', > > > 'parameters': {'number_folds': '10', > > > 'number_repeats': '1', > > > 'percentage': '', > > > 'stratified_sampling': 'true'}, > > > 'type': 'crossvalidation'}, > > > > > > ``` > > > > > > > > > Shall I just remove the first 'estimation_parameters'? The API doesn't > > > return it like this, only the python API does. > > > > > > > > > 2) when creating a task, the user needs to be able to state the > > estimation > > > procedure. This is now done by giving the task type id and estimation > > > procedure id, for instance 10 fold CV for classification has task type > 1 > > > and estimation procedure 1. > > > This is a bit hard to look up. Shall I allow people to state something > > like > > > task_type='Supervised Classification' and estimation_procedure='10-fold > > > Cross-validation'? I can do an easy lookup in the API, and it is a > whole > > > lot easier. > > > > > > What do you think? > > > > > > > > > > > > On Tue, Aug 15, 2017 at 3:57 PM Joaquin Vanschoren < > > > ***@***.***> wrote: > > > > > > > OK, I'm giving it a go. The task upload XSD is really weird, but it > > will > > > > do for now. > > > > > > > > On Tue, Aug 15, 2017 at 3:45 PM Matthias Feurer < > > > ***@***.***> > > > > wrote: > > > > > > > >> Yep, you're right. My proposal would make the API cluttered and hard > > to > > > >> understand. Creating a task object to upload would be way easier and > > > >> consistent with the rest of the API. Thanks for pointing that out. > > > >> > > > >> — > > > >> You are receiving this because you authored the thread. > > > >> Reply to this email directly, view it on GitHub > > > >> <#294# > > > issuecomment-322370079>, > > > >> or mute the thread > > > >> < > > https://github.com/notifications/unsubscribe-auth/ABpQVzGkHc8WqQAAYkZ_ > > > owaSR6Gy3tJ5ks5sYRRRgaJpZM4O2-yH> > > > >> . > > > >> > > > > -- > > > > Thank you, > > > > Joaquin > > > > > > > -- > > > Thank you, > > > Joaquin > > > > > > — > > > You are receiving this because you are subscribed to this thread. > > > Reply to this email directly, view it on GitHub > > > < > > #294# > issuecomment-322669881 > > >, > > > or mute the thread > > > < > > https://github.com/notifications/unsubscribe-auth/ACL7-iUiBnY_ > dKAe2UNOsWBI8jl6IQFFks5sYn3HgaJpZM4O2-yH > > > > > > . > > > > > > > — > > You are receiving this because you authored the thread. > > Reply to this email directly, view it on GitHub > > <#294# > issuecomment-322714244>, > > or mute the thread > > <https://github.com/notifications/unsubscribe-auth/ > ABpQVyrtumeMw9Z3vONcxZPHSWAzbQGHks5sYrQQgaJpZM4O2-yH> > > . > > > -- > Thank you, > Joaquin > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #294 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ACL7-t7H65ojsqFIYwiePjW-EgUR2mvoks5sYrTxgaJpZM4O2-yH > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpQV8ilDJXGupqID-3oRuMrqoGprjdmks5sYrtBgaJpZM4O2-yH> .

-- Thank you, Joaquin