Add new dhall rewrite-with-schemas subcommand

[Gabriella439]

Fixes #1796

This adds a new dhall rewrite-with-schema subcommand that one can use to simplify
a Dhall expression using a supplied record of schemas. Any time a
subexpression matches the type of a schema it will be replaced with the
corresponding schema.

If more than one schema matches the supplied type then the first match
(in alphabetical order) is used.

[Gabriella439]

I also tested this against dhall-kubernetes by hand (since it's too large of an example to include in our test suite). It does work (really well!), but it's slow (taking 2 minutes on my machine):

$ dhall --file ./examples/deployment.dhall | dhall schemas --record ./schemas.dhall 

let schemas = ./schemas.dhall

in  schemas.Deployment::{
    , metadata = schemas.ObjectMeta::{ name = Some "nginx" }
    , spec = Some schemas.DeploymentSpec::{
      , replicas = Some 2
      , revisionHistoryLimit = Some 10
      , selector = schemas.LabelSelector::{
        , matchLabels = Some [ { mapKey = "app", mapValue = "nginx" } ]
        }
      , strategy = Some schemas.DeploymentStrategy::{
        , rollingUpdate = Some schemas.RollingUpdateDeployment::{
          , maxSurge = Some (< Int : Natural | String : Text >.Int 5)
          , maxUnavailable = Some (< Int : Natural | String : Text >.Int 0)
          }
        , type = Some "RollingUpdate"
        }
      , template = schemas.PodTemplateSpec::{
        , metadata = schemas.ObjectMeta::{
          , labels = Some [ { mapKey = "app", mapValue = "nginx" } ]
          , name = Some "nginx"
          }
        , spec = Some schemas.PodSpec::{
          , containers =
            [ schemas.Container::{
              , image = Some "nginx:1.15.3"
              , imagePullPolicy = Some "Always"
              , name = "nginx"
              , ports = Some [ schemas.ContainerPort::{ containerPort = 80 } ]
              , resources = Some schemas.ResourceRequirements::{
                , limits = Some [ { mapKey = "cpu", mapValue = "500m" } ]
                , requests = Some [ { mapKey = "cpu", mapValue = "10m" } ]
                }
              }
            ]
          }
        }
      }
    }

I imagine that the main reason that it's slow is that it has to check every record sub-expression against every Kubernetes schema to see if they can be simplified

[german1608]

Not very fluent on lenses, but the implementation looks nice!

I was thinking that maybe we could add a cli option to override the schemas binding name, in case some users prefer to use another one rather than schemas.

Althought that is not important right now.

[sjakobi]

Looks neat! :)

Apologies for the many comments. Please just ignore the stupid ones! ;)

[sjakobi]

Oh! I wasn't aware of that IsList instance! :)

It does rely on the key order BTW, which seems okay here, because the expression is normalized. Maybe worth documenting though?!

However, schemas may have more fields than just Type and default. Those wouldn't match in this case. So better use lookup!

(And I'm wondering whether it would be better to delete that IsList instance…)

[Gabriella439]

That IsList instance is very convenient for nested pattern matching! It's also no different than the one for Data.Map.Map

[sjakobi]

The one for Data.Map seems more predictable, since it always returns the elements in ascending key order. Maybe that's something that we should adopt…

[Gabriella439]

My reasoning for preserving the order is that the IsList instance should behave like Show, meaning that the literal it produces should be valid code for creating the original value

[sjakobi]

I wonder whether normalizing the input is such a good idea. In the kubernetes example, this expands

kubernetes.IntOrString.Int 5

to

< Int : Natural | String : Text >.Int 5

In other cases it could result in more drastic blowup.

[Gabriella439]

The normalization is necessary to handle an expression like this:

{ addr.city = None Text
, addr.country = "Switzerland"
, alive = True
, name = "Alice"
}

... which desugars to:

{ addr = { city = None Text } ∧ { country = "Switzerland" }
, alive = True
, name = "Alice"
}

If you don't normalize that then the rewrite rule won't match against the { city = None Text } ∧ { country = "Switzerland" } expression

[sjakobi]

Right. Initially I wondered whether we could not check for RecordLits and instead just check that the type of the subexpression is a record type. But that would be much more expensive. For the actual rewriting we'd also have to normalize any non-RecordLits anyway.

Another idea would be to have a similar rewrite-with-types command that could replace types with those exported from some record.

[Gabriella439]

After applying the "lookup by type hash trick" the performance for the Kubernetes example improved significantly. It now only takes 3-4 seconds to rewrite the Kubernetes example

[sjakobi]

Right. Initially I wondered whether we could not check for RecordLits and instead just check that the type of the subexpression is a record type. But that would be much more expensive. For the actual rewriting we'd also have to normalize any non-RecordLits anyway.

Another idea would be to have a similar rewrite-with-types command that could replace types with those exported from some record.

[sjakobi]

👍

To fix the AppVeyor CI, I suspect that there's a Dhall.Test.Util.toDhallPath missing somewhere, but I'm not quite sure where…