Less naive rate limiter

[eccc_to_commons.git] / README

diff --git a/README b/README
index 6db096e97ae520e503ddce0889fc12ad558ffb21..9cf1c30a83e588faad07f6019815b565e2823436 100644 (file)
--- a/README
+++ b/README
@@ -10,9 +10,9 @@ distribution. In addition to coreutils, prerequisites are:
 - Bash 4+
 - Curl
 - Xmlstarlet
+- Jq
 
-This repository is sponsored by Environment and Climate change Canada and
-Wikimedia Canada.
+This repository is sponsored by Wikimedia Canada.
 
 
 Provided scripts, ordered by chronological usage:
@@ -23,6 +23,8 @@ eccc_fixer.xslt           fix upstream data XML file
 commons_rules.xsd         validate ECCC XML from a Wikimedian point of view
 eccc_to_commons.sh        transform ECCC XML files into JSON
 monthly_to_commons.xslt   transform ECCC monthly XML file into JSON
+almanac_to_commons.xslt   transform ECCC almanac XML file into JSON
+mediawiki_post.sh         upload directory to a Mediawiki
 
 
 Usage:
@@ -67,11 +69,11 @@ Here are a few examples to inspire you:
 
 Keep only monthly data:
   $ cat downloads_all | grep -B1 -A1 --no-group-separator \
-    -E '^output = ".*/monthly/[0-9]*.xml"$' > downloads_monthly
+    -E '^output = ".*/monthly/[A-Z0-9]{7}.xml"$' > downloads_monthly
 
 Remove all downloads before (restart interrupted download):
-       $ sed -n '/https:\/\/climate.weather.gc.ca\/climate_data\/bulk_data_e.html?format=xml&timeframe=3&stationID=2606/,$p' \
-         downloads_all > download_continue
+  $ sed -n '/https:\/\/climate.weather.gc.ca\/climate_data\/bulk_data_e.html?format=xml&timeframe=3&stationID=2606/,$p' \
+    downloads_all > download_continue
 
 
 1.3 Download wanted files
@@ -143,4 +145,11 @@ conversion.
 
 
 5. Upload to destination
-Not done yet.
+It's now time to share our work with the world and that's the purpose of the
+mediawiki_post.sh script.
+
+  $ ./mediawiki_post.sh "${COMMONS_CACHE}"
+
+It takes the commons cache as parameter: its file hierarchy will be replicated
+on commons. On first run, it will ask credentials for the Mediawiki account to use to
+perform the import.