Skip to content

DevicePreference: "GPU" => "GpuRequired" and "GpuPreferred" #35

Description

@yuhonglin

During reviewing the CL, Andrew (amoylan@chromium.org) proposed an interesting point: for the values of DevicePreference should we extend "GPU" to "GpuRequired" and "GpuPreferred"? The reason is that, in an ML model there may be some OPs that can not be supported on GPU. Then the backend has three choices,

  1. Fail it and returns an "unsupported" error.
  2. Run the model totally on CPU (fallback to CPU).
  3. Run the supported OPs on GPU and the other OPs on CPU (partially fallback to CPU).

Currently, we only have one value "gpu" so it is hard to tell which case above is most appropriate. After we have "GpuRequired" and "GpuPreferred", we can,

  • If the preference is "GpuRequired", then go with 1.
  • If the preference is "GpuPreferred", then go with 3.

Of course, there may be potential fingerprinting issue that the web client can directly know which OPs the user's GPU can/can't suppport.

Activity

  1. changed the title [-]DevicePreference: "GpuRequired" v.s. "GpuPreferred"[/-] [+]DevicePreference: "GPU" => "GpuRequired" and "GpuPreferred"[/+] on Mar 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions