github.com/openaddresses/machine commits | Ecosyste.ms: OpenCollective

Bumped to 1.9.2 with update to openaddr.ci.worker for Amazon EC2

f55a46c7128c9924d8d2841cbbde90f9b3ee9717 authored over 9 years ago by Michal Migurski <[email protected]>

Merge pull request #115 from openaddresses/complete-ci-worker

Completed openaddr.ci.worker for EC2

08f46543a424af3572648112bd5d11050133d8b8 authored over 9 years ago by migurski <[email protected]>

Python 3 compatibility edits

12f17f6dd53fe68886de9c2154701776b8d19b68 authored over 9 years ago by Michal Migurski <[email protected]>

Added missing parts of ci.worker chef setup

7fa67defdbf7c3cda9c03dc459a49eec819e9095 authored over 9 years ago by Michal Migurski <[email protected]>

Added EC2 hostname setup to ci.worker

0f3116118bac0e40edfafb26311098c060ce3f3d authored over 9 years ago by Michal Migurski <[email protected]>

Added Apache to ci.worker chef setup

2590c65534e37876897ccb8d74a71c148014cede authored over 9 years ago by Michal Migurski <[email protected]>

Updated worker results with complete index contents and URLs

76181b6f7b2702cb57cc7f209ef639c9771e57eb authored over 9 years ago by Michal Migurski <[email protected]>

Added tests for happy and angry ci.worker subprocesses

10f9e65c5fbd521181d810f59b3507d21b3c76d8 authored over 9 years ago by Michal Migurski <[email protected]>

Constructing worker output_url with index output

06cf859fa0e83f6adb14879476e375558909363b authored over 9 years ago by Michal Migurski <[email protected]>

Added stdout to ci.worker subprocess results

3186b41dfad797816a96c7f160d5fa1fe26f3649 authored over 9 years ago by Michal Migurski <[email protected]>

Using FQDN for openaddr.ci.worker URL

cd287d796b9069a90bf09349160988dd841f486f authored over 9 years ago by Michal Migurski <[email protected]>

Testing openaddr.ci usage outside of webhook Flask app

c3e9b57be6f2bf0e037ab539d3f6af680be32267 authored over 9 years ago by Michal Migurski <[email protected]>

Moved webhook logic into openaddr.ci.create_queued_job()

0f3a7c3d91a49b80372e7c3b57734be8a53fa124 authored over 9 years ago by Michal Migurski <[email protected]>

Including file ID in return of openaddr.ci.process_payload_files()

d52a52b74bd0c9c7fd6afb6a189edd80ad64b1b4 authored over 9 years ago by Michal Migurski <[email protected]>

Tweaked definitions of pop_task_from_donequeue() and db_connect() in openaddr.ci

80af99c62c4425cfb708b9f49b5b2316adebfb11 authored over 9 years ago by Michal Migurski <[email protected]>

Merge pull request #112 from kreed/conform_smash_case

Remove hard-coded list of keys in conform_smash_case

6474ae7776f98032b669f94432a6e194992c6359 authored over 9 years ago by Nelson Minar <[email protected]>

Remove hard-coded list of keys in conform_smash_case

conform_smash_case now matches the behavior of row_smash_case and
lowercases everything except O...

4086b08f2950877f1ea95f0c90c464f6b81da420 authored over 9 years ago by Christopher Eby <[email protected]>

Bumped to 1.9.1 with fix to case-sensitive filename errors in openaddr.conform

a7c32ba1a92246b9ab494c280f268f59a67cd260 authored over 9 years ago by Michal Migurski <[email protected]>

Merge pull request #111 from kreed/file_selection2

Fix incorrect file selection for non-shapefiles

696f759fe861b299dca24598419eea535a61d8e6 authored over 9 years ago by migurski <[email protected]>

Fix incorrect file selection for non-shapefiles

Make elaborate_filenames convert to lowercase for all types of files

2746e653411fdb8cb1ccc59433909eccef05d232 authored over 9 years ago by Christopher Eby <[email protected]>

Merge pull request #110 from kreed/file_selection

Fix incorrect file selection (issue #104)

b0b32a6862c8dba529450b237b95a8665c5f82a6 authored over 9 years ago by migurski <[email protected]>

Fix incorrect file selection (issue #104)

d6c50ba2b54b676afcebc239c9565ca1e3bb4ff5 authored over 9 years ago by Christopher Eby <[email protected]>

Bumped to 1.9.0 with new code scp behavior in openaddr.run

3fb6edda82eb81206bd5f2b298a44ed8fa5a9e58 authored over 9 years ago by Michal Migurski <[email protected]>

Merge pull request #109 from openaddresses/communicate-directly-with-ec2-#108

[WIP] Communicate directly with EC2

2d61db82ec8b19de1195d52b0939e46e3ae1caf6 authored over 9 years ago by migurski <[email protected]>

Prefixed EC2-specific command-line options for openaddr.run with "ec2-"

fa72a9b2e3b9005433a57f1c2f37e15b179a0a87 authored over 9 years ago by Michal Migurski <[email protected]>

Tweaked openaddr.run output slightly

a6456a74ea884ea6915bc29de9d4f93c8be3c36a authored over 9 years ago by Michal Migurski <[email protected]>

Added openaddr.tests.run for EC2 invokation test

55691c84598a9d2c189f823762c84bcfbd5bfc1f authored over 9 years ago by Michal Migurski <[email protected]>

Reading back complete log in event of an early failure

849a2e2bd0ca85cf4abfafcb48b0973176259cf9 authored over 9 years ago by Michal Migurski <[email protected]>

Re-try SSH connection for a period of time to wait out socket.error

205e2d7c381e5026c095651dd57d63ccf38a61a2 authored over 9 years ago by Michal Migurski <[email protected]>

Uploaded local repository instead of asking instance to download from Github

d7a1f468ab077528bd1796994344d0d81b15ecf4 authored over 9 years ago by Michal Migurski <[email protected]>

Added git package closer to its use in chef sources recipe

2380a4b53e86d2fc3bb0da2be32a14ce8642c96d authored over 9 years ago by Michal Migurski <[email protected]>

Adjusted argument order for clarity

fd977fdf1e7e8a6889d22eea58ca5f76111fba4c authored over 9 years ago by Michal Migurski <[email protected]>

Creating repository tarball in a temporary directory

754842a03944830494e46c561ca14ed8e7b57d5e authored over 9 years ago by Michal Migurski <[email protected]>

Added Git repository and branch arguments to openaddr.run

1eeb5ad1d7a9fe5eba62ffacc42096890c8944dc authored over 9 years ago by Michal Migurski <[email protected]>

Started special handling for overdue processes

df163c7bb098cfdd5522b1f549852956392a7b77 authored over 9 years ago by Michal Migurski <[email protected]>

Added Paramiko library for SSH access

58c7726995fbf0865ac90284ee7ce9230b92091e authored over 9 years ago by Michal Migurski <[email protected]>

Bumped to 1.8.0 with addition of new arguments to chef/run.sh and openaddr.process_all

1f0a7417e9d890b0b2750c9d241ec0f89d338619 authored over 9 years ago by Michal Migurski <[email protected]>

Added explicit version tag checkout to openaddr.run

1810cef3ecb0e510f1ba3eca03b03f7e32ef9512 authored over 9 years ago by Michal Migurski <[email protected]>

Made chef/run.sh backwards compatible with no-argument call

2d8b29b68c76fa04c274e798d2cfd13e31329612 authored over 9 years ago by Michal Migurski <[email protected]>

Disable SSL verification

fd9df361ba4f25af566996e9b83e87c4420b0cd5 authored over 9 years ago by Christopher Eby <[email protected]>

Merge branch 'hooked-on-sources-master'

Conflicts:
.gitignore
.travis.yml
chef/openaddr/recipes/default.rb
openaddr/VERSION
setup.p...

3ab723e2ee3de8e4e5943575c7f896329b3a9564 authored over 9 years ago by Michal Migurski <[email protected]>

Updated openaddr.ci string handling and tests for Python 3

d16cfa63bb9b3a2fed69812d9b7ddec99b50223d authored over 9 years ago by Michal Migurski <[email protected]>

Upgraded to Python 3 compatible pq 1.3

b175348c2dd0b4f99fe6d93d75804bc384825f0f authored over 9 years ago by Michal Migurski <[email protected]>

Switched to setup.py test for testing

7b0abeaeb93c68493618ded8dec86cbd6a5a1f2a authored over 9 years ago by Michal Migurski <[email protected]>

Reduced test repetition

0c3d3f0a7546d04be6f7ef4cd440f64803aa0ff8 authored over 9 years ago by Michal Migurski <[email protected]>

Added chef role just for testing purposes

46e81f70cc870b07ec9fcfbfe524ab10d90da70e authored over 9 years ago by Michal Migurski <[email protected]>

Removed database recipe from other roles, because it's expected to be a remote service

1a5df41798a1de525d2588e276c1b29b1727a866 authored over 9 years ago by Michal Migurski <[email protected]>

Added Apache proxy for Flask app

72fddff261f25fe31f769e9b7bf9757d590f81aa authored over 9 years ago by Michal Migurski <[email protected]>

Importing all openaddr.tests.* in test.py

3a1f6267cceecf8fc152070765f50379fef95bc1 authored over 9 years ago by Michal Migurski <[email protected]>

Added test_suite to setup.py

5bdbedb5cc2dab81b5d65091a09f3e4447fc1a57 authored over 9 years ago by Michal Migurski <[email protected]>

Cleaned up openaddr.tests.render

22eba3beebf30f1630141e0d2a894b3de2f66fc9 authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.util.esri2geojson tests to openaddr.tests.util

248335e1009b2262f77a383d1be3de359fa54d7d authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.sample tests to openaddr.tests.sample

47eedf91bf208ed9869b6e09fffc280a19b13fc8 authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.render tests to openaddr.tests.render

2e721c7cd9396ea95190b73397e1d8eb92bd2771 authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.expand tests to openaddr.tests.expand

ec7d3dd4e94dfd7ae46330f8bd87b925f350753c authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.conform tests to openaddr.tests.conform

d4010549daab40c3cdbc969cdc601952e171c4f7 authored over 9 years ago by Michal Migurski <[email protected]>

Moved openaddr.cache tests to openaddr.tests.cache

cb199edbff30126d8b7a9802f863a9e9b61166f3 authored over 9 years ago by Michal Migurski <[email protected]>

Moved all tests and test data inside openaddr module

7a19387ae5949ba1ca9344b69ec18809eda92d55 authored over 9 years ago by Michal Migurski <[email protected]>

Added new chef role for worker process

bf322c165d475b5ae7946c7ba1cafd76e2eb3659 authored over 9 years ago by Michal Migurski <[email protected]>

Tweaked chef recipes for database

f2dd2736da4e365e01bb87d3c482a1f4065bf190 authored over 9 years ago by Michal Migurski <[email protected]>

Added new chef role for webhook process

34e7dc2d75f6854191e42d4d3014675734476b81 authored over 9 years ago by Michal Migurski <[email protected]>

Added new database chef role for setting up local Postgres

23ada529827baa70cc8ba9d3af30a92f32a4e2f7 authored over 9 years ago by Michal Migurski <[email protected]>

Added explicit role argument to chef/run.sh and renamed chef recipes

cab3036409f02897adcf1a3090c1699c2e3a3cf6 authored over 9 years ago by Michal Migurski <[email protected]>

Renamed chef recipes

297851443b7f083d0d8750d648b753c958ea24f5 authored over 9 years ago by Michal Migurski <[email protected]>

Added explicit role argument to chef/run.sh

2596ba7b052f6116c92ca4b06364d82bb07086ba authored over 9 years ago by Michal Migurski <[email protected]>

Re-arranged setup.py and annotated requirements

faebcde1fbf17ea8d1d9257196f023abc5ccdaa5 authored over 9 years ago by Michal Migurski <[email protected]>

Packaged all of openaddr.ci based on OpenAddresses-Machine using setup.py

4841297c8412fa346de082d3375583cfb1193fa6 authored over 9 years ago by Michal Migurski <[email protected]>

Moved app under openaddr.ci module and tests under openaddr.tests

46a1237ef45b5dec15e0cc8bb5572ed40b902b3a authored over 9 years ago by Michal Migurski <[email protected]>

Configured Travis tests

683ab3debf0d3400bbf806a006375e2678025550 authored over 9 years ago by Michal Migurski <[email protected]>

Stubbed out Chef scripts

76fd73332ef042ab98a9d6dd70c2bbba8e778a06 authored over 9 years ago by Michal Migurski <[email protected]>

Add the MAGIC_OK_MESSAGE to the worker.

Sending the Zircon Princess to eternal rest.

a0520887484c72923d1ff67f13e3d9bcd58fa30f authored over 9 years ago by Nelson Minar <[email protected]>

Added dequeueing job

892629bf95fa92f7b4bcfdc79d6e000292b9236f authored over 9 years ago by Michal Migurski <[email protected]>

Very simple working worker daemon to run jobs from the queue.

9c9241838d2300748d4a0241b0e6bc13c9a5d31c authored over 9 years ago by Nelson Minar <[email protected]>

Returning a meaningful single job HTTP response

e73842b19c29e855164e1dc6e0a1d4d67e465511 authored over 9 years ago by Michal Migurski <[email protected]>

Added jobs.status column

03b5b06c966cb2c9fa3749fd1545d8c48ba28c6b authored over 9 years ago by Michal Migurski <[email protected]>

Added jobs.file_results column

2ebfdc745a27c0dbf58bce381cc6756c4a420ca6 authored over 9 years ago by Michal Migurski <[email protected]>

Cleaned up reliance on parsed job URL in pop_finished_task_from_queue()

051af25b7edca8e3bed58510a5073314cbfdcb8e authored over 9 years ago by Michal Migurski <[email protected]>

Added tests to check sequence of status updates based on task results

79443f1968fa5d396a6749f336f58f78022ac261 authored over 9 years ago by Michal Migurski <[email protected]>

Added concept of per-task file states for jobs status

24a2d3690c8906e54697a9f50030f715160d8410 authored over 9 years ago by Michal Migurski <[email protected]>

Implemented reverse-direction done queue with Github status update

Does not yet work with two-task jobs.

d5a8260cc379f83e4e6989898ad00084c3be00b7 authored over 9 years ago by Michal Migurski <[email protected]>

oops

606c7d5cfe7d11d3bec46f9567b64fed640d5531 authored over 9 years ago by Michal Migurski <[email protected]>

Clarified error response and started randomizing job IDs

390088ab804df5107791a28c42bb90f04f680963 authored over 9 years ago by Michal Migurski <[email protected]>

Changed jobs.filenames array column to jobs.task_files JSON column

a13ceb2a84c5a75819ba9bbd8049a4218baea709 authored over 9 years ago by Michal Migurski <[email protected]>

Split multi-file jobs into multiple queue tasks

3d25008c9f5f70fb0ee55cf75d72b6fb87be5113 authored over 9 years ago by Michal Migurski <[email protected]>

Stopped storing jobs.job_queue_url in database

ab1e256d0fd1ccc1111409ce25d2383798467926 authored over 9 years ago by Michal Migurski <[email protected]>

Switched to using internal database for queue

5a4b5e6fa2012fe6bd06fbb6c3099720d3e61985 authored over 9 years ago by Michal Migurski <[email protected]>

Added queue basics with new Postgres setup

9e25672a811b56b10084af4c729f6ba46a736e38 authored over 9 years ago by Michal Migurski <[email protected]>

Truncated Github status descriptions to 140 chars

3d44fb7b451afc0e3068be89f9270505dd52db94 authored over 9 years ago by Michal Migurski <[email protected]>

Switched from test-only session storage to Postgres database

e82264db6d2927b265c9ff3bd37b5ca12ee5389a authored over 9 years ago by Michal Migurski <[email protected]>

Completed mock implementation of job queue interface with tests and sessions

c2de544b6a11d006f9f3c985925c45175ff5025d authored over 9 years ago by Michal Migurski <[email protected]>

Mocked up beginning of job-queue.openaddresses.io

069cd42ec92eb129d4f195589dac3cd9cb7e47f6 authored over 9 years ago by Michal Migurski <[email protected]>

Moved status tests out of HTTMock reponse content

3f74873ba8043e483d0ad7bcbf06612bca73a839 authored over 9 years ago by Michal Migurski <[email protected]>

Peeled Github status API calls into new functions

120a950f3900ee9a9d4ceba7b10a34353790db60 authored over 9 years ago by Michal Migurski <[email protected]>

Tightened test code

4cf82829edb57fe515ca68e7e0bd6cd1cef3ef22 authored over 9 years ago by Michal Migurski <[email protected]>

Modified branch/master behavior

* For a set of commits pushed directly to master, we look at any changed files in those commits....

2322f63c74f8b72a61d254262b70e2306f5eb8f6 authored over 9 years ago by Michal Migurski <[email protected]>

Tightened test code

5b6dde842011a14ffae0ac1cf66a51b5845312d1 authored over 9 years ago by Michal Migurski <[email protected]>

Switched to whole-branch tests that no longer work on master commits

547218e0f93157f301e5ecdb20748fb199e2f67f authored over 9 years ago by Michal Migurski <[email protected]>

Pushing commit status back to Github

3342c9706086d884bb58862129bf5fc574000c4b authored over 9 years ago by Michal Migurski <[email protected]>

Added Github auth token setting for API usage

84213d9c1098fd0ee17c14ad326ee85c5dbe2cba authored over 9 years ago by Michal Migurski <[email protected]>

Added basic unit tests

002d0786f75907d152f7c6867d796d03e7605ae1 authored over 9 years ago by Michal Migurski <[email protected]>