test runner for component

[matthewmueller]

i'm building out the testing pipeline for components and it'd be great to discuss and pull in matthewmueller/component-test.

Part of the reason angular became popular was because it came with an easy-to-use opinionated test runner (testacular/karma)

[jonathanong]

i'm +1. i know TJ's life long dream is to make a component for everything like golang, but until then, it would be great included (since that'll probably mean a rewrite). we might have to rebrand as component.js or component.web

pretty sure one of our goals is to move towards small, reusable, and high quality modules, and you need tests for that. an easy way to create and run tests is important part of that goal. i know i didn't bother to make tests for a lot of my stuff because it was too difficult before component test

[dominicbarnes]

+1

I didn't hear about component-test until today, and I'm likely to start using it right away. I already use mocha, and I would love a way to automate test envs like this. I think getting something like this into core would be a great idea.

[ianstormtaylor]

i do really like the idea of component making testing dead simple, so easy that it just makes the "right" way to do something (tests, deps, small modules, etc.) the easiest way to do it anyways

[tj]

it would be nice to unify but it seems like something that could be really annoying to maintain, especially compatibility-wise, but I guess seeing as we have a little army of people going now it could happen haha

[jonathanong]

plus, i think it's pretty good where it is now.

[jonathanong]

i'm not sure where this belongs in the spec, though. @matthewmueller maybe you want to add some rules or something? dunno.

[bmcmahen]

+1 for making adding tests as easy as possible.

[matthewmueller]

@jonathanong @visionmedia I could be missing part of the vision, but to me the registry is the big piece that can be unified across languages along with client actions like install, search, etc.

I don't think that the component should try to accommodate all languages, I think component should be the home for client-side javascript libraries. That's a big enough job.

The build, the testing, the manifests (ie. component.json or package.json) will all be language specific and should probably be under a different project.

[tj]

the thing with the registry is that every ecosystem re-invents the same thing over and over, usually in ways that don't scale too well, but they all effectively do the same thing - making it work for everything would just be a tiny bit more work than building one for component. but yup the client side manifest stuff is definitely completely separate, you won't have go using package.json or anything like that, just whatever it would be (go.json for example) and a little client-side shim that formats it for the global registry

[jonathanong]

yeah i'm cool with this as long as the repo is in the component org. i think it would make a great workflow. not sure how to add this in the spec though.

[ianstormtaylor]

+1 as part of component, testing seems super good to have in core, and then yeah i think it doesn't impact the registry vision i think cuz thats just for packages

but i think that the scope for component-test should be more limited, because @visionmedia's right that we don't want to be maintaining that kind of stuff really.

i don't quite get what it does for travis, or how that relates to component test at all. that sounds like a separate module. saucelabs sounds kinda cool, but i also dont get how that works with creds. having aliases for all the browsers seems like overkill. it should just test in the default browser imo. and having things like seamonkey seems way too edge case. i dont even know what maxthon is :)

honestly, component test would solve 99% of my testing by just running mocha with phantom on my test/ folder. (and if phantoms not installed fallback to browser)

[ianstormtaylor]

component test                  # test in phantom (or default browser)
component test --browser        # tests in default browser
component test --browser safari # tests in safari

and then component saucelabs and component travis can be separate plugins that dont need to be in core

[jonathanong]

i'm cool with that. this is what most people would be doing anyways.

i'd prefer if the api were still component test browser safari or just component test safari.

[ianstormtaylor]

im fine with component test safari adding browser in is weird cuz then its a flag without the normal flag syntax

[matthewmueller]

I'm fine with separating them out too. I think @yields has a better saucelabs integration with gravy anyway. Last I checked, there's still a few fixes I made that need to be integrated though (like JS errors and reporting on ZSH is broken). I'm also for removing the crazy browsers and the syntax change.

I do feel that getting some sort of multi-browser continuous integration (travis + sauce, or whatever) is the next big step for component. If we want anyone to take component seriously, we need to have more comprehensive testing - half the components we write don't even work on IE9.

The reason jQuery kills it is because it's a drop in script that works everywhere. They owe that to their crazy test suite + test swarm. Their testing is pretty complicated though and I think we can do better.

Also, anyone know how to speed up phantom's initialization? For me it's faster to just open the browser and test than to use phantom. If the default is going to be phantom i'd like it to be faster.