Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/getsolus/ferryd

Fast, safe and reliable transit for the delivery of software updates to users.
https://github.com/getsolus/ferryd

jobs: Fix crazy bug where go GC's the Job between a function call

For realsies, without this fix, the JobEntry is GC'd when going into the
function pointer, so we...

971b625dbaee372bc741dd7dfc5ca3b993f3c674 authored over 7 years ago by Ikey Doherty <[email protected]>
Sync boltdb

Signed-off-by: Ikey Doherty <[email protected]>

a809960799e1c312f8c5012f1b601cc044be1f27 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Hook up the transit process handler

Signed-off-by: Ikey Doherty <[email protected]>

5dc50511f2e29a81dab42cc5480fc6cf0d32f40f authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Reimplement transit job for incoming uploads

Signed-off-by: Ikey Doherty <[email protected]>

dbc8c0ea132375876f1af23c67abaeed163ae100 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Simplify constructors, share Process in Execute()

Signed-off-by: Ikey Doherty <[email protected]>

3a2bf478c8b804c0a6ad827835383ed153462839 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Restore BulkAddJob + handler with new APIs

Signed-off-by: Ikey Doherty <[email protected]>

f4ca3fcf873a9bb4868d200c999427156f62e507 authored over 7 years ago by Ikey Doherty <[email protected]>
core: Ensure newly created repo has an index, even if it's empty

Signed-off-by: Ikey Doherty <[email protected]>

b5ec5c613461f4e352307d39e1f9dd0f5f0e9299 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Restore indexing operation

Signed-off-by: Ikey Doherty <[email protected]>

503be14793a068eff25f8956d9650ba3b940e1f4 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Use more granular time indices for sleep-stepping

Signed-off-by: Ikey Doherty <[email protected]>

01a5a064d9171e6c24814657a790b4fd28cae0f1 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Return to old jproc API and handle sequential decision internally

Signed-off-by: Ikey Doherty <[email protected]>

65e9b7d1512106e5e43aa70a59cf56d0cdf64298 authored over 7 years ago by Ikey Doherty <[email protected]>
ferryd: Use a specialised handler interface for deserialised job handling

Signed-off-by: Ikey Doherty <[email protected]>

6eb1bde1b0cab108cca1f36b981409a996a8fc82 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Allow jobs to describe themselves in human terms, not params

Signed-off-by: Ikey Doherty <[email protected]>

5ad9b415057b2e7a2133cef6e1cb4dfc63d02fed authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Make enum a string enum for usability reasons

Signed-off-by: Ikey Doherty <[email protected]>

23c266ab578e45c7bb8857e928edf692c161cd9c authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Use numerical ID and not a byte array in log output

Signed-off-by: Ikey Doherty <[email protected]>

df08676ea8e434a27f039075aacd44adb0d0269e authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Make CreateRepoJob a working thing again

Signed-off-by: Ikey Doherty <[email protected]>

9a209b40bdbc2bb19284680885a3f82835a2e22b authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Unfuck async (take one job only) and fix segfault in time sleepers

Signed-off-by: Ikey Doherty <[email protected]>

7aea682b12caef00bd749f4b84fc9e32711c715b authored over 7 years ago by Ikey Doherty <[email protected]>
Actually test creation of jobs!

Signed-off-by: Ikey Doherty <[email protected]>

cdea4659135d3e0833bc55cfbdab5e7dd77c4ee8 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Emit params in debug

Signed-off-by: Ikey Doherty <[email protected]>

ade9a9bcac7225214a60dd76b973d81a55f7ecf1 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Stop Golang being a snowflake with string arrays

Signed-off-by: Ikey Doherty <[email protected]>

fd6232490e81c330c4905201d35acd2d72bc95b2 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Only stop a non-nil ticker

Signed-off-by: Ikey Doherty <[email protected]>

8bd05cfe7e2c4b1ade70d600c7a146580865e19a authored over 7 years ago by Ikey Doherty <[email protected]>
Hack Manager to expose DB to JobStore, add JobStore to Server

Signed-off-by: Ikey Doherty <[email protected]>

aa48cfd3e42502e7d7b33e413163cbc2ed6aa6de authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Partially port processor to support new jobstore interface

Signed-off-by: Ikey Doherty <[email protected]>

3f2c4c1d8173228fdd2a999cd4d425dd7bcea8a0 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: job.go -> main.go

Signed-off-by: Ikey Doherty <[email protected]>

9140ed2f11b4fb42dfd2df60a95029b2924b439c authored over 7 years ago by Ikey Doherty <[email protected]>
Stop building all the old job stuff for now until we port them

Signed-off-by: Ikey Doherty <[email protected]>

0db1010731757f60102202f5cad178bd73dd48c0 authored over 7 years ago by Ikey Doherty <[email protected]>
Neuter all current job handling

Signed-off-by: Ikey Doherty <[email protected]>

af34ce7b018923fbd7caf8415a3915e4c23b4b0c authored over 7 years ago by Ikey Doherty <[email protected]>
Copyright assignment to Solus, ferryd developers

Signed-off-by: Ikey Doherty <[email protected]>

243dc912201650d46c637774156429368c2e3892 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Set first breakpoint at 0.5s

Signed-off-by: Ikey Doherty <[email protected]>

15caf8f6ec008d5cad8c3108ec135b7b529b0299 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Don't add wg in Start(), this will be done via a goroutine

Signed-off-by: Ikey Doherty <[email protected]>

0474cd9e1a922ba74b1c31043ed2cf644eeb2828 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add some handy logging within the Worker

Signed-off-by: Ikey Doherty <[email protected]>

74c9a0e2794b82ff3f7d547b42bc89feee8d587f authored over 7 years ago by Ikey Doherty <[email protected]>
worker: Simplify code, add main loop and time index code

Signed-off-by: Ikey Doherty <[email protected]>

7bb08587e8d7a8fd8b572b4af6d631397df02360 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Begin work on the new Worker implementation

The Worker is effectively our thread type, which will know ahead of time
whether it is a sequent...

75e47ec4654bcd310043473b32adae5422642542 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Keep hold of the ID when claimin an async job

Signed-off-by: Ikey Doherty <[email protected]>

06c94931580100cb69a5a2fb62cf1981e55e199c authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Be consistent in 'bucket' terminology for variables

Signed-off-by: Ikey Doherty <[email protected]>

06713f06bd37c4638d19afebf36f8932f14b9170 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Use boltdb ForEach for safe iteration

Signed-off-by: Ikey Doherty <[email protected]>

53669ac6265b9357c605b58f23ab28ffe3091611 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: "Sync" -> "Sequential"

Signed-off-by: Ikey Doherty <[email protected]>

1abae121d54e0be4ff397ca8ea0a929d78c75dd0 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Nest the buckets in case we share a DB, fix pushJobInternal to use name

Signed-off-by: Ikey Doherty <[email protected]>

dab72547644503c2ff58c030a2ea31fe0559a1e0 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Remove "JobStore" bucket unless we really need it

Signed-off-by: Ikey Doherty <[email protected]>

95f738fa4c8b74dd6021b91ee749762c3aa76fb4 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add new magic code to insert sync/async jobs

Signed-off-by: Ikey Doherty <[email protected]>

e251fc63e4be6482e05acedd6bb056f7c242df27 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Use var-style "constants"

Signed-off-by: Ikey Doherty <[email protected]>

d90f4b0ccbdf8c4b80f6cea749e46fbd2578fd46 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Make JobEntry partly opaque, stop passing job ID around

Signed-off-by: Ikey Doherty <[email protected]>

9a6d87e1cb3c7737d5b820bf4042c5188c3851f4 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Simplify jobs, hide IDs, use full transactions

Signed-off-by: Ikey Doherty <[email protected]>

895c94b32918dbe18c0803eabb1c72b64291b6c0 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Simplify setup code

Signed-off-by: Ikey Doherty <[email protected]>

4ec9de6024d083739be837fcd1042c73af93de53 authored over 7 years ago by Ikey Doherty <[email protected]>
Merge pull request #2 from DataDrake/master

So, I hear you know concurrent data structures

5714713c00574919f7bab4f895c0590a8e043264 authored over 7 years ago by Ikey Doherty <[email protected]>
make compliant fixes

35c0d117978bba7382b8615d8328044a16c3ed4f authored over 7 years ago by Bryan T. Meyers <[email protected]>
Initial pass at JobStore, JobQueue not needed

79827dbe9db9502e2785dc64c73b74e8887db66b authored over 7 years ago by Bryan T. Meyers <[email protected]>
Initial queue implementation

d291f59b09a9becb945e0fd385d94b6703d1349b authored over 7 years ago by Bryan T. Meyers <[email protected]>
Add super basic usage instructions

Signed-off-by: Ikey Doherty <[email protected]>

267fed395fe48037f2dd907f73031f9b169528c5 authored over 7 years ago by Ikey Doherty <[email protected]>
Stop using the xzed library for deltas

Unfortunately using liblzma in-process leads to absolutely staggering
memory leaks, quickly lead...

dc39d4f2a6cad9d6d70575daf531e136a155e43d authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Remove Archive helper, we won't be using this now

Signed-off-by: Ikey Doherty <[email protected]>

d3143a085ca55e9f7c32414e04536d1b88ca6069 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Add new helper to extract the tarball from an eopkg

Signed-off-by: Ikey Doherty <[email protected]>

36c368124242df5c7e930cfa3120f242ed50605d authored over 7 years ago by Ikey Doherty <[email protected]>
core: Use new libeopkg helper to write + read XZ files

We've had no luck with using liblzma directly in Go as it leads to massive
memory leaks, so we'r...

aca78a17bd2e88182020bc9cf6e65ba98bfe8622 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Ensure we pop executed jobs from the table to save memory

Signed-off-by: Ikey Doherty <[email protected]>

9818a918658ef528278d69c90e02d63f2df47982 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Fix spinning locks

Signed-off-by: Ikey Doherty <[email protected]>

17e64cc2c667996e6bb00f64470f0fc8aad9ef41 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add initial work for the delta job

Signed-off-by: Ikey Doherty <[email protected]>

82c49fa40983765bfe61bab7ecabcc49ea82d37b authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Add quick ID accessor

Signed-off-by: Ikey Doherty <[email protected]>

ea50849a5d9d0df1d62788d5125c8d3efb0217ad authored over 7 years ago by Ikey Doherty <[email protected]>
core: Add initial broken delta support

Signed-off-by: Ikey Doherty <[email protected]>

e608f9c84d4c21331ae51ec5b0a60cbfdaa310da authored over 7 years ago by Ikey Doherty <[email protected]>
ferryd: When the shop is closed stop taking customers

Signed-off-by: Ikey Doherty <[email protected]>

6c3dbd89147f06eaa3c71cec6fa59f2b63860b7a authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Make the DeltaProducer slightly lighter

We don't actually need an ArchiveReader for the old eopkg, as we're purely
interested in the cen...

fc319e83fc0bdafeb6541f17ee17ec34025e9287 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Make sure we're copying from the right tarball :]

Signed-off-by: Ikey Doherty <[email protected]>

b4786aa44c75e1f87c1bb7a5e9459112efc52248 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: We now produce deltas just like eopkg (but better)

Signed-off-by: Ikey Doherty <[email protected]>

71d54234f1cc6fa1cc5c4093273f1e2c84b640d6 authored over 7 years ago by Ikey Doherty <[email protected]>
core: Add new helper function to create delta packages for us

Signed-off-by: Ikey Doherty <[email protected]>

4decd7c84f4716adda53b874fa2005473d129cf4 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: remove unused delta functions

Signed-off-by: Ikey Doherty <[email protected]>

f6552d3d18b506f01e63598d08a1b6c4e95c3eb1 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Make Commit() return temporary path so it can be reused

Signed-off-by: Ikey Doherty <[email protected]>

2992de6b33963be539a3a66c1105562d1ef2ea15 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Mostly finish the delta Commit function

Much like the tarball magicks we'll iterate the existing eopkg file, copying
all portions *excep...

f9616e207611c30373a2a78cea1ac17c04734e5c authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Simplify function complexity in delta API

Signed-off-by: Ikey Doherty <[email protected]>

f765edc9a8761a1b7c3489c9862176247558c95c authored over 7 years ago by Ikey Doherty <[email protected]>
vendor: Resync xzed to fix segfault

Signed-off-by: Ikey Doherty <[email protected]>

d356eefa9ff420075d34ed44cfeeaf00e762d177 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Produce install.tar.xz only containing the difference

Signed-off-by: Ikey Doherty <[email protected]>

7cf56e53cd88d66aa0592cc22199475360b3925f authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Simplify archive interface, add rudimentary delta identification

The logic is effectively the same as how it's handled right now inside
eopkg, any hash-mismatche...

d020a7b9f958f9ebe8f041ec56f8b2ba83f456e9 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Begin working on delta test, require import 2 .eopkg files

Signed-off-by: Ikey Doherty <[email protected]>

3d7f27d1d12aa3e0a8a4fbe0b0d1bbf1ac566918 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Begin working on the DeltaProducer

The job of the DeltaProducer is to examine two eopkgs, and determine the
files that should remai...

d2b9be48e1c71843c2fbb31b0db774c58e398760 authored over 7 years ago by Ikey Doherty <[email protected]>
vendor: Sync xzed for reader enhancements (io.Reader vs io.ReadWriter)

Signed-off-by: Ikey Doherty <[email protected]>

ee9a871f3393e6795cb90162b7c9cdbcc1cacc23 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Open the tar reader via an xzed.Reader

Signed-off-by: Ikey Doherty <[email protected]>

f167510107b84bb7abf94fac1db9ea7155452a9b authored over 7 years ago by Ikey Doherty <[email protected]>
vendor: Sync xzed for improved read/write of eopkgs

Signed-off-by: Ikey Doherty <[email protected]>

133de2eb5e3cf51e3a35da2d69faf1a63c8f5a03 authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Begin working on a new ArchiveReader

This incomplete implementation will be used to read the eopkg's main contents,
i.e. the install....

e57dc8d4298db20928d8f38303a9b5382232f0ca authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Add support for the files.xml parsing

Signed-off-by: Ikey Doherty <[email protected]>

8ecf63347b7fae8f8bd01cc3e776f680cf340344 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add basic processing queue for delta incomings (skeleton)

This will allow us to process each set of packages in batch, i.e for a package at
a time. Curren...

a23393ca6410d6cd794edde4567ba72d0800427d authored over 7 years ago by Ikey Doherty <[email protected]>
repo: Fix broken GetPackages function

Signed-off-by: Ikey Doherty <[email protected]>

eb9725b86ff73cd033cc7fed5190b485d3ee5e21 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Fix dependency handling to ensure they run and don't lock

Signed-off-by: Ikey Doherty <[email protected]>

4d72cd4e43ff0353b7d5bd588b5e9b619afb57b2 authored over 7 years ago by Ikey Doherty <[email protected]>
Add basic delta workflow (still doesn't create them)

Signed-off-by: Ikey Doherty <[email protected]>

fc8d55e24eaaca1c889d841e93adebd52cabbff9 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add basic dependency management and delayed execution

In essence we want to keep things going as quickly as we can, so we'll
add job dependencies to t...

cf64342f94a87be6758b91dbf66c941c565da517 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Neuter the delta job for a minute so we can build stuff

Signed-off-by: Ikey Doherty <[email protected]>

aa50ffb6f434c99188c03086d55e20c23e91f816 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Hide public Job fields and return opaque Job object

Signed-off-by: Ikey Doherty <[email protected]>

e0858899a44eaec003474a1ad21c6de3c79333d0 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add non-building delta call job

Signed-off-by: Ikey Doherty <[email protected]>

cff8c5fa22c250e16109de140cc20c8553a14291 authored over 7 years ago by Ikey Doherty <[email protected]>
core: Add new methods for GetPackages + GetPackageNames

These will allow us to iterate the contents of a repository and then
fetch all of the packages f...

58a9046bfcb6f2f87c590a110d3628933e5de552 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Allow passing the JobProcessor to other jobs

Signed-off-by: Ikey Doherty <[email protected]>

c7f269cce918cc862c959bf4f19e6899c26960c9 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add bare job for deltas (does nothing yet)

Signed-off-by: Ikey Doherty <[email protected]>

dc33f260ca1b98edc90af965c44763a8eeaf1e5b authored over 7 years ago by Ikey Doherty <[email protected]>
ctl: Add client-side code to request re-delta of a repo

Signed-off-by: Ikey Doherty <[email protected]>

0533c3e862cf99f0964d4b2150489e16b2dcb11b authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Add some helper functions (untested) for delta creation

For now this is proxied via eopkg itself, which is highly inefficient
at doing this, but thats k...

88a04b772f386fd666a686c0a863fe0f123afa8a authored over 7 years ago by Ikey Doherty <[email protected]>
libeopkg: Teach the metadata index about delta packages

Signed-off-by: Ikey Doherty <[email protected]>

f2b6cb46ee814511fdcc9e57752fe317ee2c8d8f authored over 7 years ago by Ikey Doherty <[email protected]>
index: Fix complexity of function because Go Report Card :3

Signed-off-by: Ikey Doherty <[email protected]>

12f087d4de10d1d71479b3138a8976313e8546be authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Hide public member "Task" as an implementation detail

Signed-off-by: Ikey Doherty <[email protected]>

8c9617ef3bea57bd4d5fcd92c6dda27694ff6f7b authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: StatusComplete -> StatusSuccess

Signed-off-by: Ikey Doherty <[email protected]>

aec458cc654f187f99a06254fd3038dfa8a27d92 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Add metadata to each Job (Status + ID + Timing)

Signed-off-by: Ikey Doherty <[email protected]>

99e9aa8afcffed26cd6ef7b4b7852264f81faa9a authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Switch Processor to use the Job type wrapping the Runnable

The Job wraps the Runnable and now allows us to track the execution times
for each task.

Signed...

b9837079b2fe7d18a0d9e77ccebdc54fabc39bec authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Move types into main.go and begin working on definition for Job itself

The Job will hold metadata to help track the lifetime of a given task.
Chiefly these will be tra...

00e436b14984b7b1fb3e53a1681c85f8192bde75 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Rename Job to Runnable

Effectively the task is a Runnable job, which will be future managed
and wrapped by a private jo...

7264f7d14996ed5f9ed27bc8a2160ca6348dc14a authored over 7 years ago by Ikey Doherty <[email protected]>
ferryd: Add description to Jobs for future display

Signed-off-by: Ikey Doherty <[email protected]>

7377356e0a8ed7f45c22b68765d714905e8f7c10 authored over 7 years ago by Ikey Doherty <[email protected]>
Add new import command

The import command allows us to forcibly include eopkgs into the
repository without requiring a ...

e5be5ba3edca060233743be74a96fd6a55744873 authored over 7 years ago by Ikey Doherty <[email protected]>
jobs: Clean up files from the incoming directory after successful merge

Typically our merge job would've hard linked the asset into place after
the upload, which in of ...

485ae28b7f1b8ca375e24ac2c392f5969e3004bb authored over 7 years ago by Ikey Doherty <[email protected]>
Make TransitManifest processing actually work (merge to repo)

Signed-off-by: Ikey Doherty <[email protected]>

c2461d8af23dbd0c27ece0ac78fb06d4b5b8727e authored over 7 years ago by Ikey Doherty <[email protected]>