Update README

Author: devn

README.md

@@ -1,7 +1,7 @@
 # GetClojure
 
 Searchable Clojure examples programatically compiled from lots of scraping.
-GetClojure extracts valid s-expressions from various locations, runs them in a
+GetClojure extracts valid s-expressions from a set of logfiles, runs them in a
 sandbox, and captures the value and output of every s-expression. The result is
 then made searchable.
 
@@ -16,26 +16,33 @@ Go to [GetClojure](http://getclojure.org) and start searching.
 | getclojure.config | Where any globally-necessary project configuration and functions live |
 | getclojure.elastic | Houses configuration, query, and seeding from files for elasticsearch |
 | getclojure.extract | Contains logic related to parsing clojure IRC logfiles and spitting out files for later consumption |
-| getclojure.format | Logic for formatting s-expressions |
+| getclojure.format | Logic for formatting and syntax highlighting s-expressions |
 | getclojure.routes | Routes for the application |
+| getclojure.seed | Runs the full ETL pipeline from logs to seeding elasticsearch |
 | getclojure.server | The entrypoint for the server |
-| getclojure.sexp | Logic related to running s-expressions: sandboxing via SCI, formatting and printing, and production of files for later consumption|
+| getclojure.sexp | Logic related to running s-expressions: sandboxing via SCI |
 | getclojure.util | Everyone has a junk drawer. This is ours. |
 | getclojure.views.layout | Static HTML generation via Hiccup |
 
 ## Developers
 
 In order to run locally in development mode you'll need to do the following:
 
-* `pip install pygments`
-* `docker-compose up -d`
+* `pip install pygments`: There are a variety of options for keeping your python envs separate. virtualenv, pyenv, etc. Use whatever works for you. Note that you'll need python 3.X or later. In addition, a libpython file is required.
+* `docker-compose up -d`: Runs elasticsearch.
 * Download the [logs](https://www.dropbox.com/s/19yy3zn5nh8a1gr/clojure-irc-logs.tar.gz?dl=0) and extract them into the `resources/logs` directory.
-* Capture the working expressions: `lein extract-sexp-input-file`. Grab a cup of coffee. This takes about 10min on my machine.
 * Set the appropriate env vars in your `.envrc`. If you don't use [direnv](https://direnv.net/), you'll need to export `APP_ENV=development` and `INDEX_NAME=getclojure.`
-* Run `lein gen-working-sexps`, `lein gen-formatted-sexps`, and finally `lein seed-elastic`.
+* Run `lein seed-elastic-partial 25` to get the s-expressions found in the first 25 logfiles, or `lein seed-elastic-full` to run the entire pipeline across all local logfiles.
 * Start the server: `lein ring server-headless`
 * Visit [localhost:8080](http://localhost:8080) and search.
 
+## Contributions
+
+Contributions welcome. This project uses `lein test-refresh` for rerunning
+tests. Run `lein test-refresh :all` to run all tests, including the integration
+tests. If you have questions about contributing, please reach out to me on
+Clojurians Slack (@devn) or Libera IRC (devn).
+
 ## Thanks
 
 * To borkdude for providing so much great open source software to the Clojure community.

src/getclojure/server.clj

@@ -17,11 +17,3 @@
 (defn -main []
   (ring/run-jetty #'app {:port 8080
                          :join? false}))
-
-(comment
-
-  (def server (ring/run-jetty #'app {:port 8080
-                                     :join? false}))
-  (.stop server)
-
-  )