Conversation
piiswrong
reviewed
Oct 26, 2020
| help='Either rsa key path or rsa key pem must be given.') | ||
| parser.add_argument('--url', | ||
| type=str, | ||
| help='URL to webconsole.') |
piiswrong
reviewed
Oct 26, 2020
| type=str, | ||
| help='URL to webconsole.') | ||
| parser.add_argument('--username', | ||
| type=str, |
piiswrong
reviewed
Oct 26, 2020
| type=str, | ||
| help='Username of webconsole.') | ||
| parser.add_argument('--password', | ||
| type=str, |
4265c66 to
b789f2d
Compare
b789f2d to
29c0ca6
Compare
piiswrong
pushed a commit
that referenced
this pull request
Oct 26, 2020
* add csv data * fix file_type for tree model * add field 'label' to tfrecord_raw_data * fix num_parallel * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * add loss_type for run_tree_worker.sh * eliminated data path * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * rebased and modified defaults * deleted csv data Co-authored-by: zhangzihui <zhangzihui@bytedance.com>
piiswrong
added a commit
that referenced
this pull request
Oct 26, 2020
* Add job/update functionality * fix * add migration * fix * fix * fix * fix * fix * fix * add heartbeat webconsole * fix * trigger * Add migration create table * Add client side job validation * Fix create/update job * fix * Feat/datasource (#328) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Feat/datasource (#330) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * ds bug fix (#331) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * change delete job logic * feat: use full params to set default (#338) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Fix form defaults * Fix client_params & server_param inversion * feat: config replica type (#339) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * add replica workers (#340) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * bug fix (#346) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * fix pod-log pod-shell (#351) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Update server.js * remove replicas from data join jobs default * move DataSource select to ticket (#354) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell * fix: move datasource select from job to ticket Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * feat: federation config (#356) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell * fix: move datasource select from job to ticket * feat: federation config: * fix init value of boolean Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Add k8s settings default * Add datasource_meta migrations * chore: remove useless default * Pass DB env args to pod * Add new psi data join entry script * update psi data join default json * Fix bug in webconsole create job * fix: env not update (#364) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Allow injecting form default from env var * Add +x for scripts * Turn off cmd display * fix: v1.5 bugs (#376) * fix: v1.5 bugs * ticket raw data name * form data not updated when editing ticket * federation form checker * chore: remove console.log Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Fix form-default * fix: job can not submit (#378) * fix: job can not submit * Update CommonJobList.jsx Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Fix delete job error * change Role assignment for PSI data join * feat: code key (#381) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * fix: edit code key (#382) * feat: code key * fix: edit code key * feat: search param data * Update CommonTicket.jsx Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * fix: search exit condition (#383) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * feat: clone ticket and rawdata (#384) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * build DATA_PATH from DATA_SOURCE for tree worker * Full integrated test (#385) * add csv data * fix file_type for tree model * add field 'label' to tfrecord_raw_data * fix num_parallel * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * add loss_type for run_tree_worker.sh * eliminated data path * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * rebased and modified defaults * deleted csv data Co-authored-by: zhangzihui <zhangzihui@bytedance.com> Co-authored-by: Allen Shen <932142511@qq.com> Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: ADeltanp <45765741+ZhZhang711@users.noreply.github.com> Co-authored-by: zhangzihui <zhangzihui@bytedance.com>
piiswrong
added a commit
that referenced
this pull request
Oct 29, 2020
* Add job/update functionality * fix * add migration * fix * fix * fix * fix * fix * fix * add heartbeat webconsole * fix * trigger * Add migration create table * Add client side job validation * Fix create/update job * fix * Feat/datasource (#328) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Feat/datasource (#330) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * ds bug fix (#331) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * change delete job logic * feat: use full params to set default (#338) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Fix form defaults * Fix client_params & server_param inversion * feat: config replica type (#339) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * add replica workers (#340) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * bug fix (#346) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * fix pod-log pod-shell (#351) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Update server.js * remove replicas from data join jobs default * move DataSource select to ticket (#354) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell * fix: move datasource select from job to ticket Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * feat: federation config (#356) * feat(web_console): form json * chore(web_console): make form fields immutable * mask sensitive data * fix: lock version * feat(web_console): support popover on nav tab * feat(WIP): datasource * feat(web_console): use form meta to implement form convert * chore(web_console): update doc * feat(web_console): raw data form: * form utils unit test * fix logic of `fillJSON` * fix: raw data submit: * fix federation value validator * fix: image should be written into context * fix: async response on submit * fix: ticket not clone job * chore: use `useMemo` with form fields * feat(web_console): rewrite `fiilJSON`: * support two array mode * feat(WIP): datasource job form * form item `name-value` support ignore keys * incremental update form value in group type change * TODO: form submit * feat(web_console): boolean select in form * feat(WIP): datasource ticket * fix header active tab * feat(web_console): datasource and trainning * fix(web_console): ticket * update form * create ticket * job link * add `filterArrayValue` * fix(web_console): clone job * feat(web_console): stop and start job * fix: update job status * feat: job federation id filter * fix: job status * fix: package-lock * fix: prerender error * fix: prerender error * fix: boolean select; * update router of charts * fix: bool-select * feat: header dropdown icon * feat: federation heartbeat * fix: clone job * fix: heartbeat * fix: server ticket select * fix: raw data in clone * fix: delete job rpc * fix: clear formMeta on cancel * chore: remove raw_data in job form * feat: show total info of server ticket: * fix create jon derect in the index * fix server ticket pull in cloning * fix: useEffect * fix: server ticket info * chore: remove mock data * chore: fix typo * feat: disable federation with filter * fix: raw data fill: * fix raw data appState * fix: num_partitions * fix: raw_data fill: * TODO: figure out why not working in some conditions * feat: use full params to set default: * change default value when modify job type * feat: select datasource in training job * feat: config replica type: * feat update form value when update * fix: hide env value in training * Update form-default.js * fix: replica num * feat: add replica workers * fix: data source and worker nums fill * fix: replicas is null * fix: pod-log pod-shell * fix: move datasource select from job to ticket * feat: federation config: * fix init value of boolean Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Add k8s settings default * Add datasource_meta migrations * chore: remove useless default * Pass DB env args to pod * Add new psi data join entry script * Show data join metrics * update psi data join default json * Fix bug in webconsole create job * fix: env not update (#364) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Allow injecting form default from env var * Add +x for scripts * Turn off cmd display * fix: v1.5 bugs (#376) * fix: v1.5 bugs * ticket raw data name * form data not updated when editing ticket * federation form checker * chore: remove console.log Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * Fix form-default * fix: job can not submit (#378) * fix: job can not submit * Update CommonJobList.jsx Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * Fix delete job error * change Role assignment for PSI data join * feat: code key (#381) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * fix: edit code key (#382) * feat: code key * fix: edit code key * feat: search param data * Update CommonTicket.jsx Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> * fix: search exit condition (#383) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * feat: clone ticket and rawdata (#384) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * build DATA_PATH from DATA_SOURCE for tree worker * Full integrated test (#385) * add csv data * fix file_type for tree model * add field 'label' to tfrecord_raw_data * fix num_parallel * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * add loss_type for run_tree_worker.sh * eliminated data path * add num_parallel for compatibility. * Revert and add new fake feature in tfrecord raw data for tree model. This reverts commit 8ca6061. * modified tree trainer for debug purpose, will be reverted * Revert debug, add send metrics in tree trainer, shrink raw data, full integrated test supported This reverts commit 977cd45. * rebased and modified defaults * deleted csv data Co-authored-by: zhangzihui <zhangzihui@bytedance.com> * fix: replica type (#388) Co-authored-by: shenshihao <shenshihao.20@bytedance.com> * fix: const closure * fix: job type Co-authored-by: Junyuan Xie <eric.jy.xie@gmail.com> Co-authored-by: shenshihao <shenshihao.20@bytedance.com> Co-authored-by: Eric Junyuan Xie <piiswrong@users.noreply.github.com> Co-authored-by: ADeltanp <45765741+ZhZhang711@users.noreply.github.com> Co-authored-by: zhangzihui <zhangzihui@bytedance.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PSI & Tree supported, csv data added (but not tested), run_tree_worker.sh fixed.