Add support for static 3D favicons

[rcabanier]

Favicons are currently limited to 2 dimensional images.
Create a specification that allows the display of 3D content.

This feature will build upon the existing infrastructure and must not introduce new fundamental concepts or security concerns.
We will propose a very simple addition to the existing favicon syntax.

[fernandojsg]

Link to the thread started on the public IW mailing list for more context: https://lists.w3.org/Archives/Public/public-immersive-web/2018Aug/0007.html

[DRx3D]

There were two comments from the email thread that were not answered (at least that I noticed).

1. "...work with Khronos on a web-friendly version of gltf". Having worked with Khronos and Fraunhofer during the early development of glTF V2, I can say it is web friendly. Of course it could be web-friendlier; but at its basic, data comes in from a remote connection and with minimal CPU processing, goes into the GPU. I don't think you will get much throughput improvement with static geometry; however, textures and animation may have significant improvements.

2. glTF V2 is strictly Physically Based Rendering (PBR). That strong;y prefers use of the GPU to render the model. Preventing the use of shaders would put a severe limit on rendering.

[dwsinger]

As a point of information, the HEIF file format can support alpha, animation, and alternatives (and is not tied to the HEVC coding format). Not that I think in-file alternatives are a great way to go if there are out-of-file ways to express that.

[mkungla]

How would you expect future bookmark pages to work? Browser running 100-s of scripts to process individual 3D favicons as individual scenes or bookmarks would be independent scene which renders each bookmarked icon as object in bookmarks scene?

[rcabanier]

@DRx3D is it your belief that gltf is already perfectly safe to use on the web?
At a minimum, I think we need to state that there is a maximum byte size and that the files should be self-contained.
Is there a danger in the shaders that are used? Could they stall or crash the GPU?

[rcabanier]

@dwsinger Why would we use HEIF? Can it describe a 3D model?

[rcabanier]

@mkungla yes, managing all the assets for each favicon will be a burden on the browser.
Having just a model file means that browser can keep using their existing logic.

[dwsinger]

@rcabanier HEIF can encapsulate any format. So, for example, we could define how to store a popular 3D model format as the primary item. (In fact, one can store anything that has a mime type, anyway). I just wanted to make sure you are aware; I'm neither proposing nor anti-proposing. But it would be easy to define HEIF variant for which the primary item is a 3D model, and there is an alternative that is GIF/JPEG/PNG coded. Similarly there are existing definitions to say "that item is a transparency mask for this one" and so on. HEIF allows MP4 tracks for animation, as well.

[rcabanier]

@dwsinger since this is for favicons which already have an established fallback mechanism, I think HEIF won't buy us much.

However, if we want to add support for 3D models natively to the web page, I can see this being very useful.

[TrevorFSmith]

@dwsinger I like the general idea of an encapsulation format but I'm concerned that the burden of adding this level of abstraction (especially with a format that isn't supported in other aspects of most UAs) would limit update among UA implementers.

glTF is gaining tool and workflow support as well as general use in existing web apps, so if we choose a single format for 3D favicons it seems like a reasonable choice.

[TrevorFSmith]

I've renamed this Issue to make it clear that it's for discussing the static favicon path.
If anyone would like to discuss the possibility of discussing the dynamic, scriptable favicon path then please open a separate Issue.

[DRx3D]

@rcabanier: To some extent glTF safety is dependent on what you mean by "safe". It is possible to crash a browser with a glTF model. It is also possible to crash a browser with a sufficiently large JPEG. glTF does not contain code (including shader code) and by itself does not render. Except for size and animation complexity, I have not seen any glTF model that crashes browsers or otherwise leaks information. The display of the model depends on the code used to display it. Khronos specifies the shader equations and may provide sample code. There are open source implementations of rendering code (e.g., THREE.js + glTF loader) that include shader code. Their safety/reliability depend on individual implementations. I am not going to warrant that everything that can be expressed in a glTF file is risk free -- I have not studied the format with that in mind. There may be some pathological cases that fail. Also the standard continues to evolve. Also glTF has the capability of adding cameras and lights to the scene. These can cause more issues. OTOH, glTF is an open standard along with several implementations and that makes it easier to put lots of eyeballs on it.

…

-- *Leonard Daly* 3D Systems & Cloud Consultant LA ACM SIGGRAPH Past Chair President, Daly Realism - /Creating the Future/

[RafaelCintron]

I am good with having a 3D model serve as a favicon, so long as we prohibit the execution of arbitrary shaders or code. Formats that rely on a PBR-only based approach fit the bill for me.

[cwilso]

Favicons are defined by HTML - and I think you actually don't need to add anything there, as the rel=icon type isn't restricted type-wise IIRC, it will just be "now this type is recognized" (we used to support a very different set of file types for icon - pouring one out for .XBM files!). For app manifest, it's a bit harder, as "icons" is defined as an array of ImageResources (https://www.w3.org/TR/appmanifest/#icons-member), which are supposed to be type=image in terms of Request. Not sure if that will be a problem, will need to investigate.