github.com/chaoss/grimoirelab-kingarthur commits

[repositories] Add method to convert Repositories objects to dicts

fc21f3f59a253652bae9aa181719854ab47a561f authored over 8 years ago by Santiago Dueñas <[email protected]>

Create README file

Fixes #1

bf98884b1a0bba036c119a99c072265507055e07 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthurd] Update usage message

caabff746177d3a9ac3b14c508c12929bd573d96 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthurw] Update usage message

b6f32a40b6c7a0ad573f923d0da0d175428eab2d authored over 8 years ago by Santiago Dueñas <[email protected]>

Add requirements file to build Arthur

f8a073930d69d9440ab963871e79cf469a832c65 authored over 8 years ago by Santiago Dueñas <[email protected]>

Add version to arthur package

11efd9b1f160595cf9ac8f9d35f5d7ec72e4ddae authored over 8 years ago by Santiago Dueñas <[email protected]>

[server] Fix crash error when a writer was not set

f82a6640280284d44aa0c41128143d580be11619 authored over 8 years ago by Santiago Dueñas <[email protected]>

[scheduler] Avoid massive CPU usage while polling for jobs

The scheduler was set to wait 0 seconds to check whether new
jobs were ready to run. This caused...

055c877f00daf6b84ea7bffef532047e746eaaff authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthurw] Output Perceval logging messages

Currently, RQ workers were configured to output only those messages
proced by the class Worker. ...

54572fa65b7e2a79afd8d558b7f18219f5121264 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthurw] Add Arthur worker command

This ad-hoc command will replace the call to RQ worker command.
Calling this command was painful...

d77c5bdb411ed47b867dccd2bcab14732d3cb6ed authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthurd] Make configurable the server endpoint address

So far, arthurd daemon only listened to connections on
'127.0.0.1:8080' address. This change mak...

f6a7bc6a348d1984278c151970a56eaf9d6aa930 authored over 8 years ago by Santiago Dueñas <[email protected]>

[scheduler] Add delay parameter for rescheduling jobs

Using 'scheduler_delay' parameter will let the user to configure
the different delay values requ...

cc7d108fface50f87e4689be12a6fef343a26be3 authored over 8 years ago by Santiago Dueñas <[email protected]>

[scheduler] Schedule jobs using 'sched'

This commit introduces the use of 'sched', an event scheduler
from the standard library. This al...

8aaa71d2b51a0f0c1c9ff2bcb8a0982b3e155b35 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthur] Make scripts executable

30f1fcc9b7ee8ab5db7a0586c9c6e0751567daed authored over 8 years ago by Santiago Dueñas <[email protected]>

[worker] Update worker to support rq >= 0.6

This change makes that older versions of rq library will not
work with arthur.

594800feedec8b3d8ad9f93e6356a60910d8aa80 authored over 8 years ago by Santiago Dueñas <[email protected]>

[scheduler] Use 'offset' data to reschedule jobs

ac9cffed4456de2fb4d8f5b39d7fd17e5d18d335 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthur] Fix error initializing threads on daemon mode

Some threads like the scheduler thread or the writer thread
were initialized before daemonizing ...

c61215490cecbc81c14af01b3f2dda6bf9552a28 authored over 8 years ago by Santiago Dueñas <[email protected]>

Update setup.py to install arthurd daemon

04c0954e9ba8770268bacb31896b95717d1e1399 authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthur] Add Arthur's daemon

25579211a88cedf4e61ead67d9c8888d9b1fc0ff authored over 8 years ago by Santiago Dueñas <[email protected]>

[writers] Set mappings on ElasticSearch writer

These mappings set the string type to not analyzed and disables
the dynamic mapping.

7cc9d3e2cc7dbe3ccf659fae3b1d72dab300eb4b authored over 8 years ago by Santiago Dueñas <[email protected]>

[server] Add thread to write items in an output

When a writer object is given, the Arthur's server will run
a thread to write the fetched items ...

9ecb627bc63defa600c61d933fa9af8b1ae7960c authored over 8 years ago by Santiago Dueñas <[email protected]>

[writers] Add class to write items in an ElasticSearch index

3398bd3823de56532913e6487ff974cc55cb75a6 authored over 8 years ago by Santiago Dueñas <[email protected]>

[server] Add HTTP REST server

455481df44c3d3cd95ea6993f11360bb199be1dd authored over 8 years ago by Santiago Dueñas <[email protected]>

[arthur] Import str_to_datetime() from utils module

53f01b39cdc50f9bbd9b8d71621cb5145ef2c4f6 authored over 8 years ago by Santiago Dueñas <[email protected]>

[utils] Add function to convert strings to datetime objects

1362a93c70f55b59fd32fbd8c3d3527c430c6b74 authored over 8 years ago by Santiago Dueñas <[email protected]>

[jobs] Return in a job result the maximun date of the fetched items

So far, Arthur was returning inside the job result the date of
the last fetched item. Due to ite...

559148fd72a51bf2a4b897ab3657f5c36ab5d6b0 authored over 8 years ago by Santiago Dueñas <[email protected]>

[jobs] Include origin and backend data into a job result

f9997a82cc3865a634156b4600a9458943445e7c authored over 8 years ago by Santiago Dueñas <[email protected]>

[repositories] Pass backend arguments as a dict not as keywords

26ff7ca9b7c97ebc86e8ceb70ab58c94bffa95f1 authored over 8 years ago by Santiago Dueñas <[email protected]>

[jobs] Fix error generating JobResult when no items were fetched

4e07ea5e6cad143cf342409b00527e5aeebe5104 authored over 8 years ago by Santiago Dueñas <[email protected]>

[tests] Fix error testing job results data

cd5a5b30b0c2d98bd9fd1835ac4a36690e173f8b authored over 8 years ago by Santiago Dueñas <[email protected]>

[repositories] Add a read/write lock for concurrent access to data

0850d82cb6c8b5f758fd5e93aaf91a0082513dc5 authored over 8 years ago by Santiago Dueñas <[email protected]>

[utils] Add read-write lock to solve readers-writers problem

0595e509c8eb5859464b1eff3e996799f6a13985 authored over 8 years ago by Santiago Dueñas <[email protected]>

[worker] Include the job result in the job notification message

d9de863c446c9b1d87795e3a23e7c5ad95461adf authored over 8 years ago by Santiago Dueñas <[email protected]>

[jobs] Return a JobResult object when a Perceval job is finished

a39fda28e289924c1cb0368c3385c9f58a6ee607 authored over 8 years ago by Santiago Dueñas <[email protected]>

[scheduler] Fix error re-scheduling jobs that did not produce data

30e6f108bb72a8b43f5c7c59694d0d6c1850b9af authored almost 9 years ago by Santiago Dueñas <[email protected]>

[arthur] Add cache support

8cfcdec9e2f99a7c196172a28031f499c76ee30d authored almost 9 years ago by Santiago Dueñas <[email protected]>

[scheduler] Schedule jobs using the cache

6f62abe34ce8cf99f2d2a085db4536e30c68317e authored almost 9 years ago by Santiago Dueñas <[email protected]>

[repositories] Include cache parameters in repository objects

e5c755114b810c95fce20f9632bec9e4ed3b4655 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[jobs] Add support to use Perceval's cache when jobs are executed

269c2e18cc6850bbf64ecada581b9afbe876a523 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[arthur] Add option to set remote Redis databases

92840687d42fb6f2c32d124d39b4b03f4181d4ca authored almost 9 years ago by Santiago Dueñas <[email protected]>

[worker] Fix error resolving Redis connections

58c48a03bb79b3bf0d61b92ad1013af425c227d6 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[scheduler] Schedule finished jobs

fb220306866da714537054dd759db1b3c70f854a authored almost 9 years ago by Santiago Dueñas <[email protected]>

[jobs] Update jobs functions to work with Perceval 0.1.0

86f73a5f42fe49a7b98a1cbe55c025cf2dbd1434 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[worker] Add custom worker to notify when a job has finished

The notification will be sent using a pubsub Redis channel.

031dbd2c8274c90cb75c87c18026fb5073deabd9 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[arthur] Clear queue when items are read

497eac6e0ee38d7ff1b620ff2dffd97196d447a2 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[tests] Update tests to work with the latest Perceval

a2625aeafb3e45cc52f9b55d6e2df19a44ae68e0 authored almost 9 years ago by Santiago Dueñas <[email protected]>

Add setup.py file

9a06a42758617320a7b45a2fbb03d174f9be93ec authored almost 9 years ago by Santiago Dueñas <[email protected]>

[arthur] Add basic script to run Arthur from the command line

65506e2d71aca2b999a645e01de8dd382db34818 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[scheduler] Add default jobs timeout

b83c5eb395bf0287a6627165599c0faf4442eeeb authored almost 9 years ago by Santiago Dueñas <[email protected]>

[errors] Add exception to manage invalid dates

50e2d18165a4b3eedca0d91ad693d23a56ce7756 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[arthur] Add main class to manage the system

f1be3b17e475458e525af7ef9ba45191e8e1ac61 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[scheduler] Add class to schedule jobs

74983bb5620f5d95e95b73672908f4e4b0df4c86 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[jobs] Store items in the given queue

Items were stored in a queue named like the origin of the
repository. This patch fixes the error...

90a88d08cf02830d083b24a5bda73bad089c3963 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[jobs] Add 'execute_perceval_job' function

This function runs a Perceval job on RQ. The items
generated are stored in a Redis shared queue....

ac2404960fd1140c319859e59d52a35262ca38cf authored almost 9 years ago by Santiago Dueñas <[email protected]>

[jobs] Add functions to execute Perceval backends

476ddb94f503babb6cabf66715cd35c9e746e3b7 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[repositories] Add classes to manage repositories

136d30c4a61eeacac6a689ba4ed9ab7fe8a60846 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[errors] Add base exceptions

beff95cf9a6a3242693a7101cc94dc1caa3f490e authored almost 9 years ago by Santiago Dueñas <[email protected]>

Add gitignore file

a47a65e847bbc5e4b1db50a78ec2f0ec4d73de83 authored almost 9 years ago by Santiago Dueñas <[email protected]>

[tests] Set up unit tests framework

defb8a0dc1ed95cfcf573c81f0782c10fdbda871 authored almost 9 years ago by Santiago Dueñas <[email protected]>

Initial import

03dcc0480f1fd9459df4d38e44d40b0f2ec84458 authored almost 9 years ago by Santiago Dueñas <[email protected]>

Ecosyste.ms: OpenCollective

github.com/chaoss/grimoirelab-kingarthur