Mousedown event in SVG "crashes" Threepenny

[brprice]

The symptom

Consider the code

import Graphics.UI.Threepenny as UI
import qualified Graphics.UI.Threepenny.SVG as SVG

main :: IO ()
main = startGUI defaultConfig setup

setup :: Window -> UI ()
setup w = do
  pure w # set title "Buggy event position in SVG"
  elOut <- textarea
  elOk <- UI.div #+ [string "non-svg mousedown is ok"]
  elBad <- SVG.text # set html "svg mousedown is bad" # set SVG.x "10" # set SVG.y "50"
  elSVG <- SVG.svg # set SVG.viewBox "0,0,100,100" # set SVG.width "300" # set SVG.height "100" #+ [pure elBad]
  getBody w #+ [grid [[pure elOk, pure elSVG], [string "Event information:", pure elOut]]]

  on mousedown elOk $ \p -> pure elOut # set text (show p)
  on mousedown elBad $ \p -> pure elOut # set text (show p)  -- crashes as fails to parse data as a pair of ints
  --on (domEvent "mousedown") elBad $ \p -> pure elOut # set text (show p) -- look at the EventData
  pure ()

Running this we find that clicking on the "non-svg mousedown is ok" text behaves well: it dumps the click location into the textbox.
However clicking on the "svg mousedown is bad" behaves poorly: instead of doing the same as above, "crashes" threepenny. By this I mean that it reloads the browser window and we see in the console

Foreign.JavaScript: Browser window disconnected.
src/Graphics/UI/Threepenny/Internal.hs:190:24-55: Non-exhaustive patterns in Success y

The problem

Assuming that on mousedown is supposed to work within an SVG (if not, what is the alternative?), the problem is that (https://github.com/HeinrichApfelmus/threepenny-gui/blob/master/src/Graphics/UI/Threepenny/Events.hs#L64)

readCoordinates :: EventData -> (Int,Int)
readCoordinates json = (x,y)
    where [x,y] = unsafeFromJSON json

-- | Mouse down event.
-- The mouse coordinates are relative to the element.
mousedown :: Element -> Event (Int,Int)
mousedown = fmap readCoordinates . domEvent "mousedown"

assumes that the EventData are integers, which is false.

To see it is false, comment out the second to last line in the example code and uncomment the penultimate line to see the raw event data.

The workaround

One can work around this simply by writing a version of readCoordinates which returns (Float,Float).

Depending on how we want this event (and also mousedown and mouseup) to work, it would be good to either

• document that you cannot use this inside an SVG and give an alternative
• change the assumption that the EventData is integral (I worry this may break existing code though)

However, I am unconvinced that this is the correct solution, since the raw javascript event handler seems to only have integers. Unfortunately I don't understand how threepenny-gui calculates its result (presumably) from these raw integers.

[sjakobi]

Many thanks for the report and the great reproducer @brprice! :)

(For my own reference) I built it within the threepenny-gui directory, with

$ cabal build -w ghc-8.8.3
$ cabal exec -w ghc-8.8.3 -- ghc issue238.hs

Interestingly I wasn't able to reproduce the issue with Chromium (v81.0.4044.122). With Firefox (v75.0) I get the buggy behaviour you reported!

Which browser did you test this with @brprice?

I've had a look at https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent. The various coordinates included there (I'm not quite sure which ones we actually use!) are documented to be doubles now, but they previously were longs! See e.g. https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/clientX.

To me, this suggests that we should change readCoordinates to produce Doubles. It would be nice to do this in a backwards-compatible way of course, but I don't quite see how…

@HeinrichApfelmus Thoughts?

[brprice]

Tested with Firefox 76.0 (sorry - I really should have included this info up front!).

To clarify what I meant by the raw javascript returning integers: running

document.getElementsByTagName("svg")[0].children[0].addEventListener("mousedown",(e) => {console.debug(e)})

in the debug console to attach an event to the SVG text, we then see something like the following in the console:

mousedown { target: text, buttons: 1, clientX: 471, clientY: 58, layerX: 471, layerY: 58 }

compared with the EventData from threepenny-gui which gives

Array [Number 146.2833251953125,Number 16.0]

So I'm not sure why the EventData is not actually integral in this case (I don't follow how it gets computed from presumably the same underlying data).

Thanks for digging through the mozilla docs -- I was unaware that they specified coordinates to be doubles now - this makes the idea of changing the type of readCoordinates more palatable!
(Though there is the issue of backwards compatability...)

[HeinrichApfelmus]

Thanks for the report! Crashes happen at runtime when types change from long to double and there is no type checker to catch it. 😅

I think we should reflect the type change in the web standards and change the return type of readCoordinates to (Double,Double). Just changing the types is enough, unsafeFromJSON takes care of the rest.

Two questions for @brprice :

• Threepenny uses pageX instead of clientX when calculating coordinates. Could you check whether these are also returned?

threepenny-gui/js/lib.js

Line 40 in a5997e9

var x = e.pageX - offset.left;

• Could you test different values for the viewbox attribute and confirm that the coordinates returned by mousedown do change, i.e. that they refer to the whole page and not the internal SVG coordinates?

[sjakobi]

Regarding standards: https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent#Specifications seems to indicate that the latest official specification of MouseEvent (which still used long coordinates) is obsolete, and that various more recent "working drafts" use double.

I'm in favour of changing readCoordinates to return Doubles then. For convenience we could add e.g. readCoordinates' with the old type.

@brprice Would you be interested in making a PR?

[brprice]

@HeinrichApfelmus

• Firefox indeed reports pageX and pageY for the mousedown event and on this test page it seems that pageX,clientX and layerX all have the same value (similarly for the Y versions)
• Different values for viewbox (note that this makes the SVG text move around on the web page): trying to click on the same place (start of "svg mousedown") and looking at X component:
  • original (0,0,100,100) : Firefox says 326, threepenny says 1.2
  • (0,0,200,200): 321 & 1.2
  • (0,0,50,300) : 362 & 2.2
  • (10,40,10,30): 353 & 4.9
• As previous, but clicking at the start of the m in "svg mousedown" (because threepenny-gui measures offsets relative to the element - so clicking at the start is a bit pointless):
  • original (0,0,100,100) : Firefox says 359, threepenny says 34.2
  • (0,0,200,200): 338 & 18.2
  • (0,0,50,300) : 372 & 12.2
  • (10,40,10,30): 460 & 111.9
    Looks like they indeed measure screen coords rather than SVG coords.

@sjakobi : sure! I'll try to get something in tomorrow.

[brprice]

For the record, I have dug into lib.js and worked out why threepenny-gui was returning doubles to Haskell (which were then choked on) even though Firefox was giving integers as pageX etc.

This happens because we give the mousedown event data as relative to the element, and the element's offset is fractional even though the mouse position is integral. Specifically

threepenny-gui/js/lib.js

Line 39 in a5997e9

var offset = $(this).offset();

this jQuery returns fractional data for the SVG element but not the plain html one.

This is documented at https://api.jquery.com/offset/ under "Additional Notes".

[brprice]

Closing because this was fixed in cf147cd and released with 0.9.0.0 (#242).

@sjakobi, @HeinrichApfelmus : thanks for getting this out so quickly!