Skip to content

_pickle.UnpicklingError: invalid load key, '<'. #2

New issue
New issue
Open
Open
_pickle.UnpicklingError: invalid load key, '<'.#2

Description

@junminren2
junminren2
opened on Dec 27, 2023

hi, I found sth maybe needs to be revised:
the link is changed, below is correct.
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/cvo_raft_patch_8.pth
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/movi_f_raft_patch_4_alpha.pth
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/movi_f_cotracker_patch_4_wind_8.pth
!wget -P datasets https://huggingface.co/16lemoing/dot/blob/main/demo.zip
!unzip datasets/demo.zip -d datasets/
When I installed any package and ran the command below.
!python demo.py --visualization_modes spaghetti_last_static --video_path orange.mp4
!python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4
I found an error here:
Traceback (most recent call last):
File "/content/dot/demo.py", line 310, in
main(args)
File "/content/dot/demo.py", line 269, in main
model = create_model(args).cuda()
File "/content/dot/dot/models/init.py", line 7, in create_model
model = DenseOpticalTracker(
File "/content/dot/dot/models/dense_optical_tracking.py", line 23, in init
self.point_tracker = PointTracker(height, width, tracker_config, tracker_path, estimator_config, estimator_path)
File "/content/dot/dot/models/point_tracking.py", line 19, in init
self.model.load_state_dict(torch.load(tracker_path, map_location=device))
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1028, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1246, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.
I checked the track_path: movi_f_cotracker_patch_4_wind_8.pth and I guess there is a problem with the saving format of model. I really need your help, Thanks.

Activity

  1. 16lemoing commented on Dec 27, 2023

    @16lemoing
    16lemoing
    on Dec 27, 2023
    Owner

    Hi, I have found what is causing your issue. You are not using the correct download links.
    Here are the correct links (replace blob by resolve):

    wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

    I have already updated the README. Let me know if it works now.

  2. junminren2 commented on Dec 28, 2023

    @junminren2
    junminren2
    on Dec 28, 2023
    Author

    Hi, I have found what is causing your issue. You are not using the correct download links. Here are the correct links (replace blob by resolve):

    wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

    I have already updated the README. Let me know if it works now.

    Thanks for your help, it could work now. But I found the speed is very slow like this.
    image
    There is any method to accelerate it?

  3. junminren2 commented on Dec 28, 2023

    @junminren2
    junminren2
    on Dec 28, 2023
    Author

    Hi, I have found what is causing your issue. You are not using the correct download links. Here are the correct links (replace blob by resolve):

    wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

    I have already updated the README. Let me know if it works now.

    Thanks for your help, it could work now. But I found the speed is very slow like this. image There is any method to accelerate it?

    Refine target step: 0% 0/51 [00:00<?, ?it/s]/content/dot/dot/models/optical_flow.py:73: UserWarning: Torch3D is not available. For optimal speed and memory consumption, consider setting it up.
    coarse_flow, coarse_alpha = interpolate(data["src_points"], data["tgt_points"], self.coarse_grid,

  4. 16lemoing commented on Dec 28, 2023

    @16lemoing
    16lemoing
    on Dec 28, 2023
    Owner

    Thanks for your help, it could work now. But I found the speed is very slow like this.
    There is any method to accelerate it?

    Speed depends on the length of the input video. The time bottleneck is the computation of inital tracks with CoTracker. For instance, it takes ~40 sec for tracking a batch of 2048 points for a video with 120 frames.

    You can trade accuracy for speed by using fewer point tracks during inference, e.g.:

    python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4 --num_tracks 1024 --sim_tracks 1024

    UserWarning: Torch3D is not available.

    You can enable Torch3D by running:

    cd dot/utils/torch3d/ && python setup.py install && cd ../../..

    But this will not change speed much. It is mostly for reducing memory consumption for high resolution inputs.

  5. junminren2 commented on Dec 29, 2023

    @junminren2
    junminren2
    on Dec 29, 2023
    Author

    Thanks for your help, it could work now. But I found the speed is very slow like this.
    There is any method to accelerate it?

    Speed depends on the length of the input video. The time bottleneck is the computation of inital tracks with CoTracker. For instance, it takes ~40 sec for tracking a batch of 2048 points for a video with 120 frames.

    You can trade accuracy for speed by using fewer point tracks during inference, e.g.:

    python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4 --num_tracks 1024 --sim_tracks 1024

    UserWarning: Torch3D is not available.

    You can enable Torch3D by running:

    cd dot/utils/torch3d/ && python setup.py install && cd ../../..

    But this will not change speed much. It is mostly for reducing memory consumption for high resolution inputs.

    Thanks for your help, Moing. I tried inputting my own video to get a demo, but I found the program quickly finished with just output '^C'. Do you know what problems happened?

  6. 16lemoing commented on Dec 29, 2023

    @16lemoing
    16lemoing
    on Dec 29, 2023
    Owner

    Hi,don't you have any error message apart from '^C'? Can you share the video so that I can try and see if I can reproduce the issue?

  7. junminren2 commented on Jan 2, 2024

    @junminren2
    junminren2
    on Jan 2, 2024 via email
    Author

  8. junminren2 commented on Jan 2, 2024

    @junminren2
    junminren2
    on Jan 2, 2024
    Author

    Hi,don't you have any error message apart from '^C'? Can you share the video so that I can try and see if I can reproduce the issue?

    Could you please send me your private email? Thanks

  9. 16lemoing commented on Jan 2, 2024

    @16lemoing
    16lemoing
    on Jan 2, 2024
    Owner

    Hi, you can send me a message at guillaume.le-moing@inria.fr

  10. junminren2 commented on Jan 3, 2024

    @junminren2
    junminren2
    on Jan 3, 2024
    Author

    Hi, you can send me a message at guillaume.le-moing@inria.fr

    Thanks moing, have you already received my email? Thanks

  11. 16lemoing commented on Jan 3, 2024

    @16lemoing
    16lemoing
    on Jan 3, 2024
    Owner

    Not yet. Can you try again? There was a typo in my original reply. You can also try 16lemoing@gmail.com

  12. junminren2 commented on Jan 4, 2024

    @junminren2
    junminren2
    on Jan 4, 2024
    Author

    Not yet. Can you try again? There was a typo in my original reply. You can also try 16lemoing@gmail.com

    Thanks Moing, I already sent to your email 16lemoing@gmail.com. Did you received well?

  13. 16lemoing commented on Jan 5, 2024

    @16lemoing
    16lemoing
    on Jan 5, 2024
    Owner

    Hi @junminren2 ! I have received your video. It seems that your video is too high resolution ~900 frames at 1080p (reduced to 480p by default with the demo script) to be loaded all at once in memory. I suggest you try with a shorter video first. I can try and implement an online mode that loads frames one by one, to handle long videos with high-resolution frames like yours. Let me know if you are interested.

  14. junminren2 commented on Jan 24, 2024

    @junminren2
    junminren2
    on Jan 24, 2024 via email
    Author

  15. 16lemoing commented on Jan 24, 2024

    @16lemoing
    16lemoing
    on Jan 24, 2024
    Owner

    Hi! This feature has also been requested in another issue (#8) so I will try to implement it in the next few weeks.

  16. junminren2 commented on Jan 24, 2024

    @junminren2
    junminren2
    on Jan 24, 2024 via email
    Author

  17. 16lemoing commented on Jan 24, 2024

    @16lemoing
    16lemoing
    on Jan 24, 2024
    Owner

    You can specify a mask on the first frame to filter tracks (see varanus example in the README), more advanced filtering is not yet supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions