github.com/openaddresses/machine commits | Ecosyste.ms: OpenCollective

Bumped to 2.18.7 with with addition of output log to timed-out runs

1e319b51d1106ac47c6a5407090cf72248d7074f authored almost 9 years ago by Michal Migurski <[email protected]>

Removed out-of-date header link for downloadable addresses

9612856846d830fc42f56208baf63d1c55c6c1e4 authored almost 9 years ago by Michal Migurski <[email protected]>

Looking for log output of failed runs, e.g. in case of timeout

0f864480ddaad9af08dd34333b12ab0625a71e35 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.18.6 with with removal of two-hour ESRI cache deadline

a596f15a075c15aa0bf05371a75820c988df8a21 authored almost 9 years ago by Michal Migurski <[email protected]>

Removed two-hour time limit for caching ESRI responses

Need to get NY/statewide, noted in https://github.com/openaddresses/openaddresses/pull/1604

ddaf4e2407871606187a9cae2f1e6bbac38e8a78 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.18.5 with with new order for batch set runs

95bd5b8fda3995790e1930b0d8e169340af1d53d authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #313 from openaddresses/order-batch-by-time

Time-sort stream of sources for batch set

defd4b9c93897a17595a5c166700a190a123beb9 authored almost 9 years ago by migurski <[email protected]>

Added test for ci.get_batch_run_times()

248307395c8f7c1b77ec803238fa8a1c49270b2b authored almost 9 years ago by Michal Migurski <[email protected]>

Clean up

24c611cac85ec7e2efc49248db4c9c5e46618553 authored almost 9 years ago by Michal Migurski <[email protected]>

Generating a time-sorted stream of sources from ci.find_batch_sources()

d9801ef6fa369cfcae0fc548fac078dd34e7b4e6 authored almost 9 years ago by Michal Migurski <[email protected]>

Switched from Travis to Circle badge

3e9ec89dc82c041a92b3150414f70d3343e28f45 authored almost 9 years ago by migurski <[email protected]>

Bumped to 2.18.4 with with fix for memory error when iterating over CSV rows

66236acacd50071ed20ee7dfea403782316cee44 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #311 from openaddresses/fix-issue-310

Rewrite row number counting so it doesn't read the whole file into RAM

2aeca0734e25a0fb73d918154a1ba5ee7d8ed8f1 authored almost 9 years ago by migurski <[email protected]>

Rewrite row number counting so it doesn't read the whole file into RAM

This was only a bug in Python2, not Python3.
See issue 310 for details.

730490c38fbbfaad74f30b713a35f46334d566c7 authored almost 9 years ago by Nelson Minar <[email protected]>

Bumped to 2.18.3 with with fix to test typo

b9b2559afda7aa9799707679a4010d5e4c3d0b81 authored almost 9 years ago by Michal Migurski <[email protected]>

Updated docs to reflect 2.18.2

45e7173cc1f5ce09a3040ef5199e0f588bdb2486 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.18.2 with with longer JOB_TIMEOUT and API code move

21b7ccaa80e7030161f7d289127831f896ff040f authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #309 from openaddresses/decoupler-timeout-and-autoscale-#298

Expand JOB_TIMEOUT to 9 hours

bc053e1cced9e4be321499d2de95eb2dfb587e36 authored almost 9 years ago by migurski <[email protected]>

Expanded JOB_TIMEOUT to 9 hours

66647269eb8472e8628d75912b5bd7e107fcd068 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #308 from openaddresses/create-json-api-blueprint-#303

Create API Flask blueprint

c77078d7f0d3c013813d5209e232881c068fff4c authored almost 9 years ago by migurski <[email protected]>

Moved state.txt endpoints to ci.webapi

267deeddb018094f38e7e8ad85646969c5e695dd authored almost 9 years ago by Michal Migurski <[email protected]>

Removed tests of state.txt from TestHook

323a06b0bb22d342de8800c6b3f2697d5e0c4eb4 authored almost 9 years ago by Michal Migurski <[email protected]>

Moved /index.json to new ci.webapi blueprint

ff439ea444dbac83e94776e917a49550c5169244 authored almost 9 years ago by Michal Migurski <[email protected]>

Moved web app entry point to openaddr.ci.web

5b851392b0f8068208040158ccbc9b389fe8f953 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.18.1 with with new heartbeat-based metrics

eaff379f80d5442853e058895b6302dfc5729e7b authored almost 9 years ago by Michal Migurski <[email protected]>

Updated docs to reflect 2.18.0

50ae292ab25c87b7e6014aa3450293bdee72da19 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #307 from openaddresses/push-heartbeats-to-cloudwatch-#298

Push heartbeats to cloudwatch

7094ae3f91308145412a14cd044bab83f77ca274 authored almost 9 years ago by migurski <[email protected]>

Populating "expected results" and "active workers" metrics

9351e54b427f56439c96c1452ea74292ee3c7e63 authored almost 9 years ago by Michal Migurski <[email protected]>

Tightened timing of heartbeats to sleep less

3f7f1a24ba1d49369e1f306b51e971727892ba20 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.18.0 with with added heartbeats table for worker activity

69bb97e7ca81022d0cfbf934540a55fc7f90c3db authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #305 from openaddresses/openaddresses/store-worker-count-#298

Introduced heartbeats table with worker state

d8d504eb6c5ae549cdd9e1ca6d7be53a2d60d0ef authored almost 9 years ago by migurski <[email protected]>

Added worker kind to configuration and heartbeat messages

74c71445b6b364374b28f521ceae725617bb2a27 authored almost 9 years ago by Michal Migurski <[email protected]>

Added heartbeats table with failing tests

68b0d1c5e5b27a4d984579a0c8ed303d1a3f3448 authored almost 9 years ago by Michal Migurski <[email protected]>

oops

278348320f6c3fce7f326c9b47f7fd486273ed96 authored almost 9 years ago by Michal Migurski <[email protected]>

Looking at recent workers based on interval timing

6dc01ae136fa3ed9a6d08a261e6be5701c385212 authored almost 9 years ago by Michal Migurski <[email protected]>

Completed tests for heartbeats table values

9214881d30b66234d1e4f35d5e9c734991dd3c7e authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.17.1 with with addition of worker_kind in worker chef node

d4bb3744f86bd882aa8c34c7bf29902764aeef94 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #306 from openaddresses/add-worker-kind-to-chef-#298

Added worker_kind to worker chef

d074d84e4acbf570903adbed576a8186b0201eb3 authored almost 9 years ago by migurski <[email protected]>

Added worker_kind to worker chef

d005176c19370259e91fd0aad2f022aa7b4cb272 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.17.0 with with added heartbeat queue to observe active workers

43ef04d72a664be62715fcc7736a24c5455669bd authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #304 from openaddresses/add-worker-count-#298

Add running tally of active workers

82fbf3a00db7d5e06b753aa08a859f5de01555e5 authored almost 9 years ago by migurski <[email protected]>

Replaced clear_heartbeat_queue() with simpler flush_heartbeat_queue()

9cc633081a1f47a1cdf72da12eae99f6a740c82c authored almost 9 years ago by Michal Migurski <[email protected]>

Added mention of new heartbeat queue to docs

379d50c5dc3ddd14377d1b86383ade7f28d0ed4c authored almost 9 years ago by Michal Migurski <[email protected]>

Extended heartbeat interval to 5 minutes

9afa537b6ce8c6081d5b377d8cd0f4ff70fd46fe authored almost 9 years ago by Michal Migurski <[email protected]>

Cleaned up messy print() statements

915dc05182b8265bca9ae12f9edc203a4056dd6c authored almost 9 years ago by Michal Migurski <[email protected]>

Clearing out heartbeat queue regularly

0b3a58ddec2aa3fd789b5dca80ca8d59d42a88bc authored almost 9 years ago by Michal Migurski <[email protected]>

Pushed worker ID to heartbeat queue

c14bf956aa96dff40db933ecf38d7bae780a3c08 authored almost 9 years ago by Michal Migurski <[email protected]>

Added thread to wait for worker in ci.pop_task_from_taskqueue()

50074313cb5c179f35e339f090e108bf98620245 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.16.1 with with improved collections information in /index.json

67dd63a881d5fc558a38ca7bd3ff3b4e033183e0 authored almost 9 years ago by Michal Migurski <[email protected]>

Including only big-enough collections with license prose

1ed72c61081e67275f4573e684fa479e28bdb8ef authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.16.0 with with new collections links in /index.json

91b765feadce93000e8a1d9aebe47de2cf679917 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #302 from openaddresses/add-json-collection-links-#301

Add JSON collection links

f04efe15655740d952974ed5fba35907afad5d03 authored almost 9 years ago by migurski <[email protected]>

Added collections links to index.json

ea90143a7b3dadc3f03ffac298f28d35732ea7c0 authored almost 9 years ago by Michal Migurski <[email protected]>

Added failing tests for collections URLs

9da91bcdbdf8d3b5b8c268b44a0a12dfe10ff895 authored almost 9 years ago by Michal Migurski <[email protected]>

Added simple test for correct JSON content index.json

5f33e8fc87e34abcae78cf52a61d55286b2f3486 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.15.2 with with fix for Esri server bug

f89ea05d075943cb83f9b6887eb11e1f3499c769 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #300 from openaddresses/fallback_to_all_fields

Fallback to handle server errors when paginating with field list specified

267eea1098a4a9ae96381745323df51d2fe91182 authored almost 9 years ago by migurski <[email protected]>

Added tests for Washington, NM showing old Esri server behavior

58e399cb583cbf5f6b844e80a6024c7b645b008b authored almost 9 years ago by Michal Migurski <[email protected]>

Updated Lancaster, PA test to reflect new Esri probe requests

0be4dee9567f7c8d8b6ef5c66e07a182049a3d34 authored almost 9 years ago by Michal Migurski <[email protected]>

Fallback to handle server errors when paginating with field list specified.

1c51c0bbcd32035054006b50c0975e020b0b8a26 authored almost 9 years ago by Ian Dees <[email protected]>

Bumped to 2.15.1 with with zip collection bug fix

5862d82f6c6b8278394983d905a3b6b952b25f92 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.15.0 with with new spatial summary collection dir

72f25a7459cfb7787e8e60ffedb14df03ec7eed9 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #299 from openaddresses/summarize-collected-sources

Summarize collected sources

f40b43f431624d2610d2966a47bdd5346a275ee2 authored almost 9 years ago by migurski <[email protected]>

Writing collected row only when lat/lon are valid

2496bc0f68b17bd60cd7fcf326526ed1aca40d28 authored almost 9 years ago by Michal Migurski <[email protected]>

Fixed ambiguous floating point formatting errors

5e7ccf70e889500b1e68b23225f54431c3dbaebc authored almost 9 years ago by Michal Migurski <[email protected]>

Moved spatial summary work down to ci.collect._add_spatial_summary_to_zipfile()

d5ab532144caf559990a9b71ed76b9a7a7cde86f authored almost 9 years ago by Michal Migurski <[email protected]>

Calculating and writing spatial summaries to collections

a3742e8e09f9f54212094bbfe6adfb60fc425591 authored almost 9 years ago by Michal Migurski <[email protected]>

Started to track address squares

afe8f096a432038b16778ff4b6701f940108ba96 authored almost 9 years ago by Michal Migurski <[email protected]>

Parsing every line in every source when collecting

e0366a9f186575eada448f11987e97039ab7e68e authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.14.0 with with addition of new data & redirect URLs

28ca57a612f0817b2d1529f0436783e0b853b819 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #297 from openaddresses/add-latest-run-permalinks-#293

Add latest run permalinks

98d11fb60d7f93926060ac60fac9da51cb21f6cf authored almost 9 years ago by migurski <[email protected]>

Added nice_domain() wrapper to state.txt results

ceddc4b46dd29f50a7ac6a2f15c347426e9cfa2b authored almost 9 years ago by Michal Migurski <[email protected]>

Added URL nicening for data.openaddresses.io S3 usage

b2566897b79bb69e3ef965e1ad8422de4000d538 authored almost 9 years ago by Michal Migurski <[email protected]>

Added JSON index file with links

7170ba17d2aea3265554601ad177a4ef23ed0acb authored almost 9 years ago by Michal Migurski <[email protected]>

Added redirect for latest downloadable run file

11e1c20ac4f5d306d8235faa967a4609d89a9fc3 authored almost 9 years ago by Michal Migurski <[email protected]>

Added read_latest_run() for completed runs

93cfd1a442bbd480be37c3200bfbdde835035a16 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.8 with with enqueue and excerpt bug fixes

2a92538326cc82aed627f5b93744605e2daa9932 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #296 from openaddresses/retry-enqueued-sources-#294

Retry enqueued sources once

93ef58971714ed9cdcf337af3d188cea9c790aad authored almost 9 years ago by migurski <[email protected]>

Added retry in ci.find_batch_sources()

d17d13b96ac854c9d36dc99cd17615b7018009c8 authored almost 9 years ago by Michal Migurski <[email protected]>

Added failing test for ci.find_batch_sources() connection errors

cb8577c65ddcdc0f8530459f02e7cdd6d61b2b33 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #295 from openaddresses/fix-berlin-sample-len-#286

Fix missing sample for certain CSV sources

b04d3a6cc16ae9b252ce915918e3f63e4c1422d2 authored almost 9 years ago by migurski <[email protected]>

Adjusted for failing Fukushima test

c5008001b7b772f14feb730a1f3c9b15f7dec12f authored almost 9 years ago by Michal Migurski <[email protected]>

Added support for weirdly-named csv file excerpts

fd76c33160b7d7874bbd369a40daa76988e3891c authored almost 9 years ago by Michal Migurski <[email protected]>

Added failing test for Berlin excerpt

83e64ad4d74b71304367ed1f86447b01ba92716b authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.7 with with fix to ci.enqueue return values

e2051f0c77135df6c061324d5387b86784e48041 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.6 with with fix to process_one output for worker

c815d25f8f7faba226bd39c437ccf12c610adade authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.5 with with fix to UTF8 encoding command-line output

ed8f71cb10f5175ba30148017b212c66dc06dccc authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.4 with with fix to UTF8 decoding command-line args

0d98536bfd8f95035480a090b011c7653f3a77df authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.3 with with skipped csv read/write in collections

8bd27ed9671564425a5e7b993a5cf6817a2f183a authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #291 from openaddresses/speed-unexpanded-collect

Stopped parsing and writing CSV for unexpanded collection

2300bdff78752ca60553cd454077245357482bbf authored almost 9 years ago by migurski <[email protected]>

Tweaked order of operations in ci.collect.add_source_to_zipfile()

ef3affb7e8394be0a076b6173d08b73f873b90c5 authored almost 9 years ago by Michal Migurski <[email protected]>

Stopped parsing and writing CSV for unexpanded collection

c9b954da6700dd982cde10faa717e2a79d5d6671 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.2 with with fix to collection unicode bug

c280620a91f321b9c2edf3c0de69ca646ac7cda2 authored almost 9 years ago by Michal Migurski <[email protected]>

Bumped to 2.13.1 with with fix to collection address expansion

1ed14e86bdff3400aa8abe7297ca413ccbd9a927 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #290 from openaddresses/limit-expansion-to-us-#285

Limit address expansion to U.S. only

74345ab7c2d99fd848dcc5bebfcb23d2d6eb792c authored almost 9 years ago by migurski <[email protected]>

Switched to testing deployed version of Python

a134d47de6b61301bafd4e02379aae187294ffc2 authored almost 9 years ago by Michal Migurski <[email protected]>

Added detail to one collection test

132b46ea220846f66f7bd01f1441b3c798fe38e8 authored almost 9 years ago by Michal Migurski <[email protected]>

Added is-U.S. check to add_source_to_zipfile() for expansion

3d8a9ee29b02d2d028948342e7b48fad19d81175 authored almost 9 years ago by Michal Migurski <[email protected]>

Merge pull request #289 from openaddresses/fix-collection-encoding-bug-#288

Fixed Python 2/3 problem in collect.expand_and_add_csv_to_zipfile()

52fb8d8393d02806830bd8192de44eb1f87a735c authored almost 9 years ago by migurski <[email protected]>

Fixed Python 2/3 problem in collect.expand_and_add_csv_to_zipfile()

0b5775c6f2d07c0e831c0aeffa7dea1deb15e071 authored almost 9 years ago by Michal Migurski <[email protected]>