Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/peermaps/ingest

Convert osm pbf files into a format that can be used in peermaps.
https://github.com/peermaps/ingest

4.0.1

967038b31e121fe39acb8fe95d0a209917dfbb30 authored over 2 years ago by substack <[email protected]>
update for latest eyros and available_parallelism

89535c8831f9c65fc691efa9bfcd5f4e6682f749 authored over 2 years ago by substack <[email protected]>
4.0.0

502d953a99ae85e293279ec2e740ecc71fcf9ce4 authored about 3 years ago by substack <[email protected]>
guard against empty insert recursive loop in the optimizer

b3613b8778e9d18ecd51d10dcb8e0aa3a93ece96 authored about 3 years ago by substack <[email protected]>
additional depth limit checks in the recursive subdivision in the optimizer

348fb00cadd953592db34641e33fb15a24be6d5f authored about 3 years ago by substack <[email protected]>
check to stop recursion if there are too many large spanning records

420c57fd640d220a2e55361d994c94315c3a693c authored about 3 years ago by substack <[email protected]>
multi-threaded optimizer

a85b647b4f5aa63ebd51be53a68cf185bc9a3754 authored about 3 years ago by substack <[email protected]>
remove debug print

8e3914629819522a90670e2a9e431504d3314a12 authored about 3 years ago by substack <[email protected]>
better optimization params

ed89089fb0eb3c66742677a4886838e74b442def authored about 3 years ago by substack <[email protected]>
recursive optimizer gets down to 1.5M in the test, the best yet by far (previous best 2.4M)

73bd7dedb8067e6d129a74a83d4b280664110d2e authored about 3 years ago by substack <[email protected]>
stub for recursive subdivision

9ab547fa49edbc801919cf529a55685b6a1c5b2f authored about 3 years ago by substack <[email protected]>
build trees manually

f07497cde3e1aefd4ef313c7a0db0f4cf48ea977 authored about 3 years ago by substack <[email protected]>
3.0.1

11fb1bdb0a70911e6db48c9f1f03c49e62d9a874 authored about 3 years ago by substack <[email protected]>
update usage for --optimize

23fcabd195c54c0cdd29fb269c0159d2a258ed4e authored about 3 years ago by substack <[email protected]>
optimizer rate size fix to align

ce34498bfb6405f542d52c879271cf93f830e8cb authored about 3 years ago by substack <[email protected]>
more space for numbers

4ef768292dde2717a99c4a62e97e25823affa7b4 authored about 3 years ago by substack <[email protected]>
3.0.0

dda9abdbe7b9ad12e833976aaa137b80e3fff3dd authored about 3 years ago by substack <[email protected]>
use crate for digit grouping

c69b7652f4ca44a747ee1cf503c6ade052ab3195 authored about 3 years ago by substack <[email protected]>
write progress info with underscores between each 3 digits

b7dba63713a4c0c18d008661642659fc0ca6bbce authored about 3 years ago by substack <[email protected]>
progress bars on scan and optimize

9221941c96a063d9f85386477a4e18532d93ef2c authored about 3 years ago by substack <[email protected]>
keep memory usage of skip set small by calculating how many boxes a given feature intersects

f455063df788d2e02c4c15d97ac409ededde98db authored about 3 years ago by substack <[email protected]>
much better optimizer

775f739e535254a422a5efe1d0fe51eec18a08ea authored about 3 years ago by substack <[email protected]>
structure for a more thorough optimizer

c8b042d09d586d0b011bd4ce0ad74fe65c69bf7c authored about 3 years ago by substack <[email protected]>
2.2.0

874d33ae260cebda7aef6e910c862a4a1b5fba16 authored about 3 years ago by substack <[email protected]>
using eyros 4.3.0

d14b3d0ae2a8b122e53707c21bfa22762222556a authored about 3 years ago by substack <[email protected]>
separate optimize phase. --inline_max_bytes and --ext_records

de94346e213cf32e3e99aaa6d40972300aea64fe authored about 3 years ago by substack <[email protected]>
2.1.0

27a1d55668db7e69da6712c78399aa1cc29530b5 authored over 3 years ago by substack <[email protected]>
usage in readme

e8a36dd30e0854e54abaaee9a6649e1372eb45b4 authored over 3 years ago by substack <[email protected]>
defaults in help msg

27aaa3e592fa1ff8cfcad88b25f77913584eaa25 authored over 3 years ago by substack <[email protected]>
use latest eyros

1ceb338bb666b75efad8911d45a2f64c95c0c250 authored over 3 years ago by substack <[email protected]>
--optimize param

74989b01963dc35b278d37525f5bd83cfbd92bff authored over 3 years ago by substack <[email protected]>
fix help message option text for --no-ingest-{node,way,relation}

476e04378ee325af23ee13c2ef682e153a932bda authored over 3 years ago by substack <[email protected]>
optimize param

a39a9949da4120eeb0b77032c42b9b97243f65ab authored over 3 years ago by substack <[email protected]>
updated readme usage

8cf9a4c1e87deecbdfb1cc7e0cedd59d081e0e93 authored over 3 years ago by substack <[email protected]>
2.0.0

cf0327a9d145542473588bb2c7f169d097a3d2ff authored over 3 years ago by substack <[email protected]>
updated ingest test

87b76ed5edcf9a0b265aa2ecb67e20c74029e69a authored over 3 years ago by substack <[email protected]>
pass through db options to eyros and list default options again with --defaults

0a001d826bd88b3ca13806c280e2f33f901a87f9 authored over 3 years ago by substack <[email protected]>
use crates version of georender-pack

cb19ba72606609c00d6ce3806048ead4435b0fe1 authored over 3 years ago by substack <[email protected]>
only insert into way deps if the relation ref table contains the way id while denormalizing relations

31ceb9d3ef92dc98fe736c63094cd3d382ff0755 authored over 3 years ago by substack <[email protected]>
with_capacity for vecs

c9685e43c7f939ee543fd1891b71b82b4c23d38e authored over 3 years ago by substack <[email protected]>
serially process nodes then ways then relations for a bit lower memory use

e8db41f06809af2273bc900603a213df68fa581d authored over 3 years ago by substack <[email protected]>
remove old dir

5103491221f303b265db474c6d5b41954a404e04 authored over 3 years ago by substack <[email protected]>
using hashbrown::HashMap

9c885542cc07cd3e430d77613b1340bebd2851d4 authored over 3 years ago by substack <[email protected]>
twice as fast with jemalloc

4a87654a12f899772b0102d5050d1ec52b59620c authored over 3 years ago by substack <[email protected]>
using osmpbf-parser 1.3.0

e10779f89ebafa3baca6774d5afe04b9a5ddf6f3 authored over 3 years ago by substack <[email protected]>
explicit offsets for nodes in ways and relations scanning

ac9c448b1cdef27bd0a285cf5017e60b0d6323f9 authored over 3 years ago by substack <[email protected]>
item slices in channels for ways and relations

6f9e2b44a5b7ec073c4760e7b49d132494e9eb4f authored over 3 years ago by substack <[email protected]>
fix bug with joining parsing threads

73bd531328733f683b93abd60b1010c016d8783f authored over 3 years ago by substack <[email protected]>
individually switch off nodes, ways, relations

7f5843a63ba7d9458af0a90c2cbee056d236ee40 authored over 3 years ago by substack <[email protected]>
turn off {node,way,relation} ingest separately with args

8b128f5ac45ddaf91e08e2b735b8768a0ee773bd authored over 3 years ago by substack <[email protected]>
move ingest fields around and remove old parts

d59e005e49737ae9453457557a13fd084835322e authored over 3 years ago by substack <[email protected]>
only scan for used way offsets in relation denormalization

985a5f1a70b862e8d99111d1e41128d2adbee832 authored over 3 years ago by substack <[email protected]>
remove unused counts

8886c854c799b80bd42e3f38cb495ecc89e3adb1 authored over 3 years ago by substack <[email protected]>
join on scans and sort offsets for partial gets

cde8b07d8976777697ac8666d9094999d74896bd authored over 3 years ago by substack <[email protected]>
add one to denormalize mutex counters

30b7f5b6073bfe8ab000934fdd56732eecaecfca authored over 3 years ago by substack <[email protected]>
--{channel,{way,relation}_batch}_size

81aeb275ecc62b21c60e8224e2a525dd92517558 authored over 3 years ago by substack <[email protected]>
sync eyros more often

5fc44ed61502f1fc3010938fa8b048841a13f7c1 authored over 3 years ago by substack <[email protected]>
parallel initial scan

d667a7306b231d03ffcc5fc1a37cd6471cb13577 authored over 3 years ago by substack <[email protected]>
osmpbf-parser dep

a2636f59fbb58f3c55ef2ae5b5d73055c1f41aff authored over 3 years ago by substack <[email protected]>
works with 1_000_000 relations at a time

589b7b0919bbda8c1a896803c8353f0766649037 authored over 3 years ago by substack <[email protected]>
relations but it's slow

e9eb2f49ee96df7d07b67ce0bc58f6a99050db48 authored over 3 years ago by substack <[email protected]>
nodes using scan technique

04be0d8d7e1a6f96835e8c25089efe5f44bb8466 authored over 3 years ago by substack <[email protected]>
new technique scanning the whole file multiple times works really well

86bb0e2bbdd90f610100c015bd87845822f18393 authored over 3 years ago by substack <[email protected]>
better nproc and batch channel send model to not buffer so much inside each thread

e6c7a2ec2da1657ff7e27f9203c5f3746d15f261 authored over 3 years ago by substack <[email protected]>
cache mutex for offsets

2d5a2ca026d2dedabc4f7bb5be0783c8008a048a authored over 3 years ago by substack <[email protected]>
fix recv iteration

2cc08b3f6e101efa2f6852394876cdcbcbe86694 authored over 3 years ago by substack <[email protected]>
close channels properly

389d314932ffae9dadcaaa65244640c438280cf4 authored over 3 years ago by substack <[email protected]>
use the cache

c27b2de98392be0a1c39d73d839e2a5b2cc13e0d authored over 3 years ago by substack <[email protected]>
item cache

02afbcdda3701d551198e144fea499b5da5e669f authored over 3 years ago by substack <[email protected]>
spawn new file handles on the same file

45da376f0012021908acc89d8d42355262e6b632 authored over 3 years ago by substack <[email protected]>
using osmpbf-parser and scanning to denormalize

fb91c5d032b3ef037dcd9fe602dc081abd741bc7 authored over 3 years ago by substack <[email protected]>
no-monitor flag properly

69d8c28e8b451d711fb0998224d603243078136f authored over 3 years ago by substack <[email protected]>
--no-monitor flag

2a0e8487db84e39d1909f7c3d7b9c04a60384b22 authored over 3 years ago by substack <[email protected]>
possibly fix issue with double prints at the end

e692e15a0ce5439f318f3bf5fb4145c3b84fb0a6 authored over 3 years ago by substack <[email protected]>
_ or - delimeter for osmxq field arguments. print default osmxq field values

14bb742932ac56ca5cb2970f1036725f34c5a3d2 authored over 3 years ago by substack <[email protected]>
update for osmxq 4.0.0

74f40c89fd7d56ef7f31be7681541873366281d3 authored over 3 years ago by substack <[email protected]>
updated for the new append format and parse underscores

8fe1edcbe19dc4b897ce4a3b00a16d8785dc8541 authored over 3 years ago by substack <[email protected]>
id_flush_{top,max_age} args

fa3ef5b5a03cfdb0f62f2139b6a792e217638d09 authored over 3 years ago by substack <[email protected]>
updates for quad top and max age

a56570bf33e6fb25e9666a6dd26cb9f6995c513c authored over 3 years ago by substack <[email protected]>
depend on osmxq 2.0.0

f9ff5ba1bcc45593d3459973ac6ba840e8b9bda2 authored over 3 years ago by substack <[email protected]>
bigger decode queue

81c16c23c1cd74197969f414345bc2ff76d2bec3 authored over 3 years ago by substack <[email protected]>
get fields from args

184ef8273d56a2725b12bf79381d2494a320abb3 authored over 3 years ago by substack <[email protected]>
osmxq published version

b231181ec655d1fadb81a3e558490b55c6c04ff2 authored over 3 years ago by substack <[email protected]>
more space for rate formatting

e4a4c6f97a12290125cf747af76a636452066b2c authored over 3 years ago by substack <[email protected]>
ref calculations fixed for relations

39a2c607ecb047c4248346eb8eeeb0fe71d8d20d authored over 3 years ago by substack <[email protected]>
possibly fix relation ref issue

0be6dca9f120c287ee619dac5f441d1644b43855 authored over 3 years ago by substack <[email protected]>
integrated stateful reporting

add006e1a0ffa521e6489787c016d3677d06f2db authored over 3 years ago by substack <[email protected]>
progress struct

a3c9f89cda064ed5b301a00007cff77d09df52f7 authored over 3 years ago by substack <[email protected]>
untracked record file

e9c1ada00582b683297f497fd89c5d52dc81ebd3 authored over 3 years ago by substack <[email protected]>
flush at the end of the pbf ingest

127fee305fe56459832af975dcfc062c10186822 authored over 3 years ago by substack <[email protected]>
ported to osmxq

65cd4f0df2d77242168ebb0beebb5039d54121d7 authored over 3 years ago by substack <[email protected]>
partition the keyspace for iteration

cedaf75ab8649958759b96a262ba9abd74ba328e authored over 3 years ago by substack <[email protected]>
handle more errors

749340bea8f8202b8bdfd26e843b5ba114fa04c4 authored over 3 years ago by substack <[email protected]>
works with threads but spawn_local() is faster in tests so going with that

7b060102fc941e93395de27f10f60cedb95291c6 authored over 3 years ago by substack <[email protected]>
better reporting

dd281cdd4d9451bfcd6fb5052c24bd4abc4166a5 authored over 3 years ago by substack <[email protected]>
swap out leveldb for rocksdb

dca53a278431f4af7d4c3d5eb85d5daf69f58450 authored over 3 years ago by substack <[email protected]>
trailing newline on termination

feb8c1221de9379525a54bea54166250435049e0 authored over 3 years ago by substack <[email protected]>
1.0.2

29570e8b14b3bb26172a1d47da5021d58f405108 authored over 3 years ago by substack <[email protected]>
help and version in readme usage

fd07f121056924ba831ab752d3fb94dad5e74497 authored over 3 years ago by substack <[email protected]>
--version

f3a9c5aa379b212c3cc52b83c94f76df12c46040 authored over 3 years ago by substack <[email protected]>