Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/pelias/geonames

Import pipeline for geonames in to Pelias
https://github.com/pelias/geonames

Merge pull request #36 from pelias/map-feature-codes-to-layers

Map feature codes to layers

8229f0f6a2678cad77704f0927a726c3239e7359 authored almost 9 years ago by Julian Simioni <[email protected]>
Merge pull request #30 from pelias/testing

Testing

5e6ebea221a809ace700cc49b4065a5e1e8967f6 authored almost 9 years ago by Julian Simioni <[email protected]>
Update end to end test expected values with new layers

dbfa3abd7e771e956d031488b15196a061498c17 authored almost 9 years ago by Julian Simioni <[email protected]>
Add layer mapping stream

This stream adds a layer property to raw Geonames records (before they
are turned into Pelias Do...

8da216fac07201966ea7d4b2f4a72810fb4b643e authored almost 9 years ago by Julian Simioni <[email protected]>
Use layer mapping stream in import pipeline

467bf247d591ac59e18d9ce201ef38816cb02a44 authored almost 9 years ago by Julian Simioni <[email protected]>
Take Document layer from layer property if available

Assuming the layer mapping stream has supplied a layer property, this
change to the doc generato...

cf3fed17428b15bb3bf634b3fd23fb80d822ff8a authored almost 9 years ago by Julian Simioni <[email protected]>
Update expected functional test output with category data

9ea39c6e5c543a50277a6171fc0d6992566c379b authored almost 9 years ago by Julian Simioni <[email protected]>
Use console.error to print functional test errors

This allows them to show up in tap-dot output

be69b4ab6857334fd3b7ff6faed99435d6538d20 authored almost 9 years ago by Julian Simioni <[email protected]>
Revert "removed support for category mapping since it's deprecated"

This reverts commit f04bcff22b7e2110bcc7efb03859a2f0bf0cf15a.

cf2bdceaf74fdd7c58f5972ce94354a6e46becab authored almost 9 years ago by Julian Simioni <[email protected]>
Add entire metadata dir to gitignore

ca367d9397fe9d61f11b4144ed786c7da9c2851b authored almost 9 years ago by Julian Simioni <[email protected]>
Download metadata after installation

853ee15ac48ad8ece352b6f654ee4526eb2f73c0 authored almost 9 years ago by Julian Simioni <[email protected]>
Only require contry info when its needed

This allows the download command to run on its own, with no meta files
present in the repo initi...

e434752444844a66d8cf32b09e5f0f3b656d3d4c authored almost 9 years ago by Julian Simioni <[email protected]>
Download metadata as the pretest step

ec3a56c0aaae599fbd51419e86edad0b4ff118a7 authored almost 9 years ago by Julian Simioni <[email protected]>
modified meta file path to start one level higher

f2eeaabce3edaa4a0b27fb35422caf987f02af8d authored almost 9 years ago by Stephen Hess <[email protected]>
Remove all metadata files

020b18f7c81f8590bc6f358ca6f8507d63f32152 authored almost 9 years ago by Julian Simioni <[email protected]>
Move task files to lib

3b50bd46e5f2e2bfc79cc1c9163f19c31a49dd3a authored almost 9 years ago by Julian Simioni <[email protected]>
Set up env variables for parallel travis builds

9adebeccf07940c1105ea7eeaa859ed8f7f63870 authored almost 9 years ago by Julian Simioni <[email protected]>
Cleanup require statements

16d78c4ec222bb37089b47b4bd6ccd2a8b51ae0c authored almost 9 years ago by Julian Simioni <[email protected]>
refactored to inject stream instead of filename for easier testing

dd9992e2e4f98bb098874a2cd95d4c9383cf6aa6 authored almost 9 years ago by Stephen Hess <[email protected]>
removed `ext` parameter since it's never used

5a482dc154aeec72b6f18670418cf500cf0060e4 authored almost 9 years ago by Stephen Hess <[email protected]>
removed `featureCodes_en.txt` references since it's unused

a17de28050a2f4f9822ceb3ad3721c17eaf2e513 authored almost 9 years ago by Stephen Hess <[email protected]>
removed `timezone` support since it's unused

78498fe75b3268abd2d28cd8eef5f36acb09c1d1 authored almost 9 years ago by Stephen Hess <[email protected]>
Accept optional config option in importer

a1ba398098fda3842486d64fae198626a6f8d1d0 authored almost 9 years ago by Julian Simioni <[email protected]>
Add tape and tap-dot packages

a7de698d859efd5c269bed58d72cd209e7d88288 authored almost 9 years ago by Julian Simioni <[email protected]>
Run unit and functional tests in travis

c53fa91b945328ecebf978e670a8f6775ee4bd6d authored almost 9 years ago by Julian Simioni <[email protected]>
Add test command

96e8da4318b2983913354c500590b2f0a40c50b2 authored almost 9 years ago by Julian Simioni <[email protected]>
switched from `mkdirp` to `fs-extra` for standardization

7a7aa4db8e503271e1f67580cd708eabd798b9c8 authored almost 9 years ago by Stephen Hess <[email protected]>
Add separate unit test NPM script

9f192dcc8bb0ca141b442ba17226dae2663f77b9 authored almost 9 years ago by Julian Simioni <[email protected]>
Add empty main test file

a757704660399d84ad8c35044d35fceb6dd485ab authored almost 9 years ago by Julian Simioni <[email protected]>
Rename unit test file

a750d46b6a5655676e4472fc3a5664fe9c80daa4 authored almost 9 years ago by Julian Simioni <[email protected]>
Use document mapper stream from pelias-model

c0aaa207e3a256546a011c1e4043e36f575f85b1 authored almost 9 years ago by Julian Simioni <[email protected]>
Add endStream parameter to import process

95b8be5f7188cdd21e70273f60d01222881bc8e0 authored almost 9 years ago by Julian Simioni <[email protected]>
Add functional test

An end-to-end test of nearly the entire import pipeline using the small
Singapore datafile.

a0d231d6fc733838a85451be7de9e42acb386107 authored almost 9 years ago by Julian Simioni <[email protected]>
Add deep-diff module

05642ceb34bd0d4f3bccae47ca612e3c661f97cd authored almost 9 years ago by Julian Simioni <[email protected]>
removed support for category mapping since it's deprecated

f04bcff22b7e2110bcc7efb03859a2f0bf0cf15a authored almost 9 years ago by Stephen Hess <[email protected]>
Move adminLookupMetaStream to separate module

e46d825eceb421c4983a112db26c69314b29c7d8 authored almost 9 years ago by Julian Simioni <[email protected]>
removed `pad` dependency, switched to `lodash` for standardization

87d456f683dcbed8ca01f062a6f5c96995aeb8b4 authored almost 9 years ago by Stephen Hess <[email protected]>
Refactored to pull pelias Document generation out to separate file

This is part of an on-going effort to increase testability and find a common interface for Pelia...

36ffd26bdc89e072be050df2dca823c490f47862 authored almost 9 years ago by Stephen Hess <[email protected]>
actually include `peliasDocGenerator` this time

b03c049cc33e8fad77f3a4bf513963b928526094 authored almost 9 years ago by Stephen Hess <[email protected]>
Use new function name for doc generator

c316a0757defcde769be70b5d8af5c51ef0ccb75 authored almost 9 years ago by Julian Simioni <[email protected]>
Upgrade to pelias-model 1.1.0

dde01a443a17ad2ff874736e4248caa9d93596c5 authored almost 9 years ago by Julian Simioni <[email protected]>
Set up precommit-hook

6f3b313243df1869341ee70c9dbc9dadf934eb97 authored almost 9 years ago by Julian Simioni <[email protected]>
renamed `createPeliasDocGenerator` to `create` for convention

dea0e6c55bc3e7fa7ea5e0c19d19e54a22d6e888 authored almost 9 years ago by Stephen Hess <[email protected]>
Move doc generator stream to lib/streams

The test file too

2e7ebcde708dd7c39356f9a3936d3ddf02f0cfb6 authored almost 9 years ago by Julian Simioni <[email protected]>
Add the rest of our standard TravisCI config

4c8e89a8f1822ad2dbed435450af46c404cf5495 authored almost 9 years ago by Julian Simioni <[email protected]>
Use container builds

ed32560b0bea91c40579575bb6c4f8e1322e9211 authored almost 9 years ago by Julian Simioni <[email protected]>
Merge pull request #29 from pelias/update_travis

Update travis

d0730eee10b7b845ca62613a6c66c288662b2c40 authored almost 9 years ago by Julian Simioni <[email protected]>
Remove Flowdock notification

016160af5c8379de162c7888d6fd08cbe173630a authored almost 9 years ago by Julian Simioni <[email protected]>
Test against Node versions 0.10 and 0.12

2e1315da83ef1a3bf5f7a61d29feea8486193fce authored almost 9 years ago by Julian Simioni <[email protected]>
Merge pull request #27 from pelias/model_1_0

Support pelias-model 1.0

b8bf83239a978450a883a7bb17713dbc0a322a25 authored almost 9 years ago by Julian Simioni <[email protected]>
Upgrade to pelias-model 1.0.1, remove carat version specifier

85b806ecb416fa62d3cbdc2a15562cd49e2a5171 authored almost 9 years ago by Julian Simioni <[email protected]>
ran over jsonlint to format for readability

fd236b9417f781aaa5f5a5a7d1f787b97da7ccb1 authored almost 9 years ago by Stephen Hess <[email protected]>
Set source and layer on all documents

For now, layer is always set to venue, to stay consistent with the old,
but incorrect behaviour.

ef052d695e7488588d5d83556d4730a74bcba195 authored almost 9 years ago by Julian Simioni <[email protected]>
Use pelias-model 1.0.0

cb6ef34281be01357066c5545c0f88510461263a authored almost 9 years ago by Julian Simioni <[email protected]>
Merge pull request #23 from pelias/ngram

migrate suggester from FST to ngrams

0b913d231393415a96c4f70035ebe67681716b03 authored over 9 years ago by Peter Johnson a.k.a. insertcoffee <[email protected]>
2.0.0

71d2a0f1d40b8a73f7e1d105d8f02c1782df501b authored over 9 years ago by Peter Johnson <[email protected]>
migrate from FST to ngrams

903e43e739ec1bdc8692e274a6679b28bd35a5f0 authored over 9 years ago by Peter Johnson <[email protected]>
semantics: rename shingle->phrase

00ad99d654969f6baa64529df53373bc4d359e56 authored over 9 years ago by Peter Johnson <[email protected]>
hack shingles

458dda442945edcdd573351e7f85640be226b690 authored over 9 years ago by Peter Johnson <[email protected]>
disable suggester pipeline

35b12dda635ace00aa88889549fef1ab48193a7e authored over 9 years ago by Peter Johnson <[email protected]>
Merge branch 'feature-taxonomy#20'

metadata/category_mapping.json, task/import.js
-Map Geonames feature codes onto the Pelias cate...

b1d73ba057bfd019c1783b232939b820ce8dae86 authored over 9 years ago by Severyn Kozak <[email protected]>
Address minor pull-request comments.

task/import.js
-Log `addCategory()` errors.
-Use `hasOwnProperty()` for greater readability.
...

eb1f9809e8c6bc4a9f2d6bca266aa0520dff3683 authored over 9 years ago by Severyn Kozak <[email protected]>
accomodation -> accommodation

metadata/category_mapping.json
-Typo in a category... oops.

362877a070bdef301bc34e996ff265d2d48bbeba authored over 9 years ago by Severyn Kozak <[email protected]>
Add the nightlife category to CSNO (casinos).

45a566177b99a8c60d30adabb032a3f54a33eac0 authored over 9 years ago by Severyn Kozak <[email protected]>
Add industry to category sets with industry:agriculture

metadata/category_mapping.json
-Some category sets with the `industry:agriculture` category
we...

4b61ad0689ad3a39187a319a68c8b5ff56301a17 authored over 9 years ago by Severyn Kozak <[email protected]>
Category map: academic -> education

metadata/category_mapping.json
-As per
https://github.com/pelias/geonames/pull/22#issuecomment...

7219cce49bcb024b5754415c79aec193caba300c authored over 9 years ago by Severyn Kozak <[email protected]>
Add the finance category to some category mappings.

metadata/category_mapping.json
-Some of the Geonames categories also fall under the category of...

1eb1d1f1078fd72c9dd67ca3bfe4633dd0235cc5 authored over 9 years ago by Severyn Kozak <[email protected]>
Category map: agriculture -> industry:agriculture

metadata/category_mapping.json
-`agriculture` was mistakenly used as a top-level category in
s...

0881b1edf7f549e7b849f518ff1017eda3b05ab0 authored over 9 years ago by Severyn Kozak <[email protected]>
Set model categories in the importer.

task/import.js
-Use the `category_mapping.json` file added in 4663087 to map
geonames records'...

aa0338796291451565a071306c2a5f5a03a02c3e authored over 9 years ago by Severyn Kozak <[email protected]>
Rename taxonomy_mapping.json to category_mapping.json

"taxonomy" is less clear than "category".

46630873c340ad5bd2d94d744aaf0f8a5e41dc11 authored over 9 years ago by Severyn Kozak <[email protected]>
Convert taxonomy file from YAML to JSON.

taxonomy.yaml -> metadata/taxonomy_mapping.json
-YAML was initially used because it's much easi...

fff7cfd638dd5d4c89b8c04eeea0f8702a2cf3b3 authored over 9 years ago by Severyn Kozak <[email protected]>
Remove unmapped categories.

taxonomy.yaml
-Remove any GeoNames categories without corresponding categories
in the Pelias t...

520b2cdc0d438cac7762b844e5d7ecf3a0092d6c authored over 9 years ago by Severyn Kozak <[email protected]>
Tweak the taxonomy mapping.

taxonomy.yaml
-Perform a second pass over the GeoNames taxonomy, tweaking some
mappings added ...

4328ea32111b37b2066be7c0152bb09ef34689a3 authored over 9 years ago by Severyn Kozak <[email protected]>
Add a prototype taxonomy mapping.

taxonomy.yaml
-Add a first iteration of the geonames-to-pelias taxonomy
mapping.
-Currently a...

4f47082e2b7d339f9245e817ad16976533e315cf authored over 9 years ago by Severyn Kozak <[email protected]>
Merge branch 'selective-admin-lookup#21'

task/import.js
-Fix #21.

df8bcd952467b7888cec5f93afb2a152625a1186 authored over 9 years ago by Severyn Kozak <[email protected]>
Add a documentation comment to adminLookupDontSet.

task/import.js
-Document the `Transform` stream that prevents the admin-lookup
from setting ce...

95f9409bdd99500a69274f11650641f8d32f8ed9 authored over 9 years ago by Severyn Kozak <[email protected]>
Document admin-lookup configuration.

README.md
-Document the `pelias-config` option that enables the
admin-lookup, as per 3acebd4.

9e32dd2f1f117a4d1c247f8addbb15b4ab9983b8 authored over 9 years ago by Severyn Kozak <[email protected]>
Make the admin-lookup opt-in.

task/import.js
-Since the admin-lookup is a form of advanced usage and requires
additional set...

3acebd42ac06120ba562b24270b37c1b76213d7d authored over 9 years ago by Severyn Kozak <[email protected]>
Add more fcodes to noNeighborhoods.

task/import.js
-The latest geonames import identified two more fcodes that
shouldn't have thei...

29594cf7c06dea3f7b8bc65e6dd8a71444cae58f authored over 9 years ago by Severyn Kozak <[email protected]>
Disable admin-lookup setters via _meta.

task/import.js
-Take advantage of the feature introduced in
pelias/admin-lookup#15, and disabl...

7368a93d1476d08fab6c40a8a658b3cc9af525f1 authored over 9 years ago by Severyn Kozak <[email protected]>
Selectively set admin values based on feature-codes.

task/import.js
-Define `adminValueFilter()`, a Transform stream that removes
admin values from...

c3281505e6ed95f8babc99b3c7a5dface2d7cc0d authored over 9 years ago by Severyn Kozak <[email protected]>
Add bin/pelias-geonames as an npm script.

package.json
-Make the importer executable runnable via `npm run` to make it
usable via `pelia...

53b4f7e6586af3f50fb1a4dc38368217f7cdc5b0 authored over 9 years ago by Severyn Kozak <[email protected]>
package.json: make pelias-model version fuzzy.

c830ec4a8b52b2e24c82965b4b3401ed67e86b08 authored over 9 years ago by Severyn Kozak <[email protected]>
package.json: v1.0.3

ae48435a0eeac2ac34336f7f1a5f98e930667ac6 authored almost 10 years ago by Severyn Kozak <[email protected]>
Merge pull request #19 from pelias/admin-lookup

Integrate pelias-admin-lookup (#18).

41537ac5c819f28a5623746f1593fcf878fe17db authored almost 10 years ago by Severyn Kozak <[email protected]>
Integrate pelias-logger.

bin/pelias-geonames, task/(download, import, resolvers).js
-Replace raw `console.log()` and `co...

026484b43042af497fbf9dd37c0da8c3c5662b35 authored almost 10 years ago by Severyn Kozak <[email protected]>
Integrate pelias-admin-lookup (#18).

package.json, task/import.js
-Add `pelias-admin-lookup` as a dependency.
-Pipe `Document`s thr...

6bc0bea8f2094aaf49120cb9acdff6fd41aef5d3 authored almost 10 years ago by Severyn Kozak <[email protected]>
bump dep

99d1216df00f71fcc9c170867d1e53b8984faff0 authored almost 10 years ago by Peter Johnson <[email protected]>
Merge pull request #17 from pelias/multiplier

adding population data

5e5360a1d50b56f148cd4ff06236b98370255fc9 authored almost 10 years ago by Harish Krishna <[email protected]>
updating package.json

db196955027626ab2369810e2871f95c1242fc9f authored almost 10 years ago by Harish Krishna <[email protected]>
only set population when its > 0

946d66967011f9ea3a13b147158fa01d7b2ea31e authored almost 10 years ago by Harish Krishna <[email protected]>
using a different weighted suggester-pipeline

46187134f167a0d8de15ed46ac705b2b83951350 authored almost 10 years ago by Harish Krishna <[email protected]>
adding population data

197c150b4b12c855a79fc2bbf0aca76aa2a981f1 authored almost 10 years ago by Harish Krishna <[email protected]>
v1.0.2

fdbc5a9c18a6bc85a8df3498e3b4b849790eae85 authored almost 10 years ago by Severyn Kozak <[email protected]>
package.json: remove unused dependencies.

6facb0a371deabe97a1c37078f2194d3c585efde authored almost 10 years ago by Severyn Kozak <[email protected]>
Resolve #16. Migrate importer to pelias-model.

task/import.js
-Create elasticsearch-bound objects using the `pelias-model`
package's `Documen...

01f411c2d09221dce1fea13e68159ea94ffe8330 authored almost 10 years ago by Severyn Kozak <[email protected]>
Revert "package.json: remove unused dependencies."

This reverts commit d491ab00947b53add8933d4f1193b11270c057b5.

package.json
-The utility that I...

9a4dd52f52a55bd6105eb10b44c3871c33311f1c authored almost 10 years ago by Severyn Kozak <[email protected]>
Merge pull request #15 from pelias/refactor/jshint

Refactor/jshint

3e820367f6fa844f4d0324c9a4d9f192fb1bae21 authored almost 10 years ago by Severyn Kozak <[email protected]>
Remove unused code from task/import.js.

task/import.js
-Remove `noop` and `countStream` from the module's exported
function, as neithe...

9d791623d01a11ed87b6796f47ceeec1c9267d8f authored almost 10 years ago by Severyn Kozak <[email protected]>
package.json: remove unused dependencies.

d491ab00947b53add8933d4f1193b11270c057b5 authored almost 10 years ago by Severyn Kozak <[email protected]>