github.com/openvenues/libpostal commits | Ecosyste.ms: OpenCollective

[openaddresses] adding some of the new Mississippi sources and city of Cape Girardeau, MO

d1bcced70605f05d50155a6b0261b3c2e9c23a4d authored almost 8 years ago by Al <[email protected]>

[fix] don't write formatted addresses in the ways-only data set unless the formatter returns non-None value

5d73aa1295eb0672348d355d614b3fa1d5a88c84 authored almost 8 years ago by Al <[email protected]>

[parser] adding chunked shuffle as a C function (writes each line to one of n random files, runs shuf on each file and concatenates the result). Adding a version which allows specifying a specific chunk size, and using a 2GB limit for address parser training. Allowing gshuf again for Mac as it seems the only problem there was not having enough memory when testing on a Mac laptop. The new limited-memory version should be fast enough.

b76b7b852778d076335ad840e2ebffe8da233aed authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Muskogee, OK

ba4052c9ba119cf28cf56a374525d1c1893c3407 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Orange County, NY

2704708f4751888a96689224853efe66ba66849f authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Polk County, NC

da62fb62bab5b02252a7681cd4095d45f2d0cf3c authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding city of Salina, KS

ce21635b00d5d3de1348659e8d884f8b2c5c93f2 authored almost 8 years ago by Al <[email protected]>

[fix] override_country_dir

b4437848c46e8d14444d462d238792c4fe51b45c authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Tippecanoe County, IN

69351cad9845ae046a9f1e3e133c79219b56843c authored almost 8 years ago by Al <[email protected]>

[addresses] more classmethods

6b8b6982aa47f031b24d9a284cdc53a1646bab99 authored almost 8 years ago by Al <[email protected]>

[addresses] making most of the methods on AddressComponents classmethods if possible so they can be accessed easily for sources not using OSM polygon lookup, etc.

f7c8a6309303f0518cc185f78d6d2455a60895a3 authored almost 8 years ago by Al <[email protected]>

[openaddresses_uk] adding OpenAddresses UK as a data set. No lat/lons but it does have addresses, cities and postcodes

702901608b7535c8789b4318ee572b404990ddfe authored almost 8 years ago by Al <[email protected]>

[addresses] making postcode before {suburb,city} more likely in the UK for #39

375f7b168469a8255746f4f835897e4e405e055e authored almost 8 years ago by Al <[email protected]>

[openaddresses] use override_country_dir config option in OA address formatter

a5d8700df3960e54b9d7433357e85524cb0f3a4d authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding override_country_dir and country codes for Puerto Rico and French dependencies

0890c712e200e7a8dc0f8f840bb62409cc0d9ad3 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add override_country_dir in Puerto Rico

c80b771f948b885694f0f28d978a709b6bde2b30 authored almost 8 years ago by Al <[email protected]>

[openaddresses] remove OSM boundaries from East Peoria

dbc5d6b8669eaa90a371f187be413274c4229bdb authored almost 8 years ago by Al <[email protected]>

[openaddresses] ignore unit containing Fl in DeKalb county

0d4c08d536b7ca4a110f6dbdb30ddad425dab218 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Kalamaria, Thessaloniki, Greece

45e71a21bbc85e27a22aaf950403a9eb10337cee authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Henry County, GA

26f5c403d3cce31e02d905f928b4b7ab60533fc3 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Gwinnett County, GA

f6e9cbf8a052631c272c1c27e45a166ac74b70b1 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Cobb County, GA

b9424c6c69a3c0721d2b3f14518f2fcc4c47054e authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding unit to Fayette County, GA and adding a field map for the cities + no OSM boundaries

357af3d465cacffdafb3e929c88c3be11b9aee57 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding DeKalb County, GA

c71fe9afbf67cdb4e456f27e40a9eb76a91f6d4f authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Fayette County, GA

412dd65d87bbe0b69770324190a9c0ef4b5cb6d1 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Tillamook County, OR

e3cff74908b1ae5e9d9619814cc8ccc6497e22eb authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Clayton County, GA

a7813dda16c3bc0bb91c7e409e18d3758cc1a07c authored almost 8 years ago by Al <[email protected]>

[addresses] fix for Colombian house number formatting if the second regex group is not found

f507f2bb3ebb37a319b7b3a64c21d12ac2b69826 authored almost 8 years ago by Al <[email protected]>

[addresses] Chinese and Colombian house number regex changes

64d0783e73612ad3da1e21070a2b1fe10de97383 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Chinese name for Wuhan, OSM uses Chinese / English for the name

7d699c52b8a181ec29fe36733ad378df94b189da authored almost 8 years ago by Al <[email protected]>

[fix] typo

68afed1658ba23c26ad8df49707261d3d40b949a authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Laval, QC

fdb07d78989c5681ede12e8130b966f66faa26b4 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Moore and Montgomerey counties, TX

c744edce12fca50a6aa27578859f2070dd5708ea authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Vernon County, MO

49fe1db613cdb95bc534fd98b921c794e46ae6c5 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding city of Monroe, MI

d4de170c94545b279c43f2da9c5f85550c8f5832 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding positional args so OpenAddresses ingestion can be run only for specific countries, subdirs, or individual files.

d0679294bfc7fb87fcfb94930f8da90e9d7e1d38 authored almost 8 years ago by Al <[email protected]>

[parser] check for non-null prev/prev2 before creating tag-based features

e39d4d2f00c5a331e649731da27ff59f16ad6186 authored almost 8 years ago by Al <[email protected]>

[fix] removing include

182d60b623027c0965e58ee594b9121f57896e06 authored almost 8 years ago by Al <[email protected]>

[fix] ignore fields in Kauai containing \n

6097eacfef64d860b3592dd39ceb91acbdc37406 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Kauai and some component additions for Maui

033e8dbb581d677677b0e74ae0a208d4f74c915b authored almost 8 years ago by Al <[email protected]>

[fix] district field for Wuhan data set

fa7446deb6f1204676f64ec981fb3a0436abfb4e authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding city of Medellín, Colombia

f006bba34510605e73e828277e24e2d61455eb52 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding new Oregon counties

2d59450a51e76dbefeecd3fa8485225a2cfdd481 authored almost 8 years ago by Al <[email protected]>

[addresses] strip phrases like "# 123" off of English street names if they follow a thoroughfare/post-directional phrase whose expansion does not contain highway/route

79c2429bba5c8ba2885c4d1ff8dbd02e81ff33ea authored almost 8 years ago by Al <[email protected]>

[openaddresses] Del Norte Couty, CA

de05292b66889c47312700e1f4c5a42035426218 authored almost 8 years ago by Al <[email protected]>

[openaddresses] Eaton County and Tecumseh, MI

93768b7ba542494ceb1a6281e5d61816ae8b298a authored almost 8 years ago by Al <[email protected]>

[openaddresses] LBC

08c68317293812759f6c97393e6f3c912119d94b authored almost 8 years ago by Al <[email protected]>

[openaddresses] city of Flower Mound, TX

a2fcac4909bc341fbc1fa4cfc715b77dd6839efd authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding new BC district data sets

1d705e80da27f381df276eae45de3ab686ab5ac8 authored almost 8 years ago by Al <[email protected]>

[fix] using size_t instead of int in address_parser/address_parser_train

6a079e86b3cfc05a291b858a57fb8d136e1949ea authored almost 8 years ago by Al <[email protected]>

[parser] using separate arrays for features requiring tag history and making the tagger responsible for those features so the feature function does not require passing in prev and prev2 explicitly (i.e. don't need to run the feature function multiple times if using global best-sequence prediction)

8ea5405c20e281ffccf0af0c1407b7fa3ded400b authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Warren County, OH

ae85e3c0a0f1954b2d57c0eb1e1a832c944c59af authored almost 8 years ago by Al <[email protected]>

[parser] using new zeros API in averaged_perceptron.c

715520f681dbb763baf7106988a988ee10056a1c authored almost 8 years ago by Al <[email protected]>

[addresses] do not exclude # from sampling in Spanish

5444b722cb3a18fc5b5d77b03eb923f98f32dffd authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding a few house number phrases as well in Colombia

f76faafd8c0ca3ed2cdfd3be05b2942788fc973f authored almost 8 years ago by Al <[email protected]>

[addresses] using the number dictionary for abbreviations in house number phrases as well

adfdc06d147de1b896a55c4b7480d2d221efa289 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding random formatting to Colombian house numbers that match the {calle}-{building number} format

7cab6758097d2511b9cdfd47167d644e7145af16 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding country-specific validators and doing no validation on house numbers in Colombia

146412f4f8595cdf8ba4278754645faf3a064ab5 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding OSM boundaries for Stearns County, MN

0e10aa6f4689886595cbb4f9194621170d05ff51 authored almost 8 years ago by Al <[email protected]>

[openaddresses] Adding city of Sioux Falls, SD

5a31513092cb036c6321a3a835278d65980907d5 authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Bogotá, Colombia

64e62cac32e8d1940f97c492149e78e3e853416b authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Commerce City, CO and creating an alias for the simple unit regex for reuse

4f128579d6f79d73af48888c913e910e6feacc1c authored almost 8 years ago by Al <[email protected]>

[utils] string_replace_char does single byte/character replacement, new string_replace to do full string replacement, again using char_array for safety, string_replace_with_array function for memory reuse

b88487f633a1f45174841bf5fa94dbf74e7a76c5 authored almost 8 years ago by Al <[email protected]>

[parser] adding phrase features for category, unit, level, entrance, staircase, and po_box phrases from the libpostal dictionaries, excluding phrases which match the toponyms dictionary (e.g. US states that can also be found in street/venue names, useful for expansion but not here), if the current token is part of both an address dictionary phrase and a component phrase derived from the training data, use the longer of the two, or both if they are the same length

da856ea5c34f3d1c7ac14c7b266e93c40d6ca773 authored almost 8 years ago by Al <[email protected]>

[addresses] allowing neighborhood components to be passed in

5b616dfb57d4569b2f699d076d842e6b5a896c68 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add city of San Luis Obispo

e7d8577ad7a3d9e061c585164033803b83440b6a authored almost 8 years ago by Al <[email protected]>

[openaddresses] add Cumberland County, NC

d6281648dcc56aea80837df5d6e30c2725992998 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add city of O'Fallon, IL

1631c25ad0b0d381e0639e439aa686f27e5dc043 authored almost 8 years ago by Al <[email protected]>

[openaddresses] add city of Scotsdale, AZ

4c4147f4656253b123cfed99b735cce6b8481c3b authored almost 8 years ago by Al <[email protected]>

[openaddresses] adding Pickens County, SC

df76cde1e7f8c03a2e04deadbf13b64d0c43722f authored almost 8 years ago by Al <[email protected]>

[parser] phrase search with address dictionaries should not use the language given at training time since it's not currently available at runtime (without pulling in the language classifier, which may be warranted at some point, especially if the model can be made smaller/sparser)

c380b3e91b85aaff0313df3367695990700a9005 authored almost 8 years ago by Al <[email protected]>

[api] include some of the new components in default address_components for the libpostal expansion API

a3e51db32d32d54063398b48e4dad7eeaaaf6f34 authored almost 8 years ago by Al <[email protected]>

[gazetteers] adding ADDRESS_PO_BOX component

32fb483e96d71a41a8c828ca178ebf773385f056 authored almost 8 years ago by Al <[email protected]>

[fix] var name in address_parser_train

ba0ccc82a3df5c33a04d8eedd7e36e81b547deff authored almost 8 years ago by Al <[email protected]>

[utils] fixing key_type in hash_get, adding int64_double map

0196fe8736742cbdf07fdd27aa71c7c69fe00b52 authored almost 8 years ago by Al <[email protected]>

[fix] that didn't work, set log level to CRITICAL

be6f48f1091a0d25a989eaf3372b0ffc1d281b3d authored almost 8 years ago by Al <[email protected]>

[fix] prevent Shapely from logging to console

26bf617a062a654718cf10a2afa8188fafdd201c authored almost 8 years ago by Al <[email protected]>

[transliteration] adding no-args option for transliteration_rules script

a0b508caf6bb30d6b500e8e8d8bf0e30640d8e8f authored almost 8 years ago by Al <[email protected]>

[fix] paren

8abfa766fd6ca98c5a8e0608d8fef52bee0e83ce authored almost 8 years ago by Al <[email protected]>

[fix] lower probability of name:prefix

06003dfbb0cc30fd867a2db00bebd9e90d880e0e authored almost 8 years ago by Al <[email protected]>

[fix] var name

92b34f6af4a7e8072491f6baee8e2632a953ee1a authored almost 8 years ago by Al <[email protected]>

[fix] config

ca79342636e6cbb4a4a5762e38b39cef8db9847c authored almost 8 years ago by Al <[email protected]>

[parser] adding long-context features which help classify the first token in the string by finding the relative positions of a) the first numeric token and b) the first street-level phrase like "Ave" or "Calle"

8eafc5730bb7a06c52b8cc7ba838f9f41d837713 authored almost 8 years ago by Al <[email protected]>

[neighborhoods] base parser config changes for new prefix/first_match options

08976c772eba8428a351848cdc3fe16f9a8a8a0a authored almost 8 years ago by Al <[email protected]>

[neighborhoods] add neighborhoods that are not the top match occasionally

64673c28752fa021f57984e45fc79c548a0a071f authored almost 8 years ago by Al <[email protected]>

[neighborhoods] add name:prefix in admin boundaries and neighborhoods (used often in e.g. Germany), use alternative/language keys as well

b99e31ca1784b9fd185656c6c9053f9c9c2bab28 authored almost 8 years ago by Al <[email protected]>

[neighborhoods] don't add point with same name as existing OSM polygon

614479aee1cacaacfcb7e994df88d8e8eaf9590d authored almost 8 years ago by Al <[email protected]>

[fix] OSM neighborhoods index check

854ac853a9fecd878f8926769a36d0b08c314f60 authored almost 8 years ago by Al <[email protected]>

[fix] var

a416a314faea9c9edeac3c26d5c22a3b84cdf125 authored almost 8 years ago by Al <[email protected]>

[phrases] fixing trie suffix search

56f68e4399c98fa9f3776d59dbc5de9c94345fa5 authored almost 8 years ago by Al <[email protected]>

[fix] poly.context

5bbc0e15d77a8e7827723ffd4ef6a7e9d704a9ac authored almost 8 years ago by Al <[email protected]>

[osm] mapping admin_level=11 to suburb in Germany, admin_level=10 to suburb in Berlin

1ee0f1fe0d8a3cde03e7d182c704561ad184ef92 authored almost 8 years ago by Al <[email protected]>

[fix] set component to suburb on OSM neighborhoods index

b9c24867d7cec6922d5601ab45c8dfec45939d25 authored almost 8 years ago by Al <[email protected]>

[neighborhoods] components are now pre-calculated by CTH index

072a838fde562ac12d69b7e6fc9cdf42ef5899b3 authored almost 8 years ago by Al <[email protected]>

[fix] rm

c91a0bdb91f73a2b0a16c680a180eb385b0a0711 authored almost 8 years ago by Al <[email protected]>

[fix] remove print

949c10ab22c8c8ab2d22ebf8f70e277a754e25df authored almost 8 years ago by Al <[email protected]>

[neighborhoods] logging, moving OSM/CTH before Quattroshapes for easier testing

738bd7b525f9493520108ee76720490dd2ef84b6 authored almost 8 years ago by Al <[email protected]>

[fix] move

67f69ce6ce3f89505d37c585bcc09baf8cec588a authored almost 8 years ago by Al <[email protected]>

[osm] neighborhood polygon reader

6d580f4c87f7c82056e0e8f890be80717fdaab34 authored almost 8 years ago by Al <[email protected]>

[neighborhoods] adding ClickThatHood config to whitelist/specify what kind of polygon is specified in each file. Adding OSM neighborhoods (ways/relations where place=neighbourhood to reduce ambiguity) as the highest priority, followed by CTH/OSM, CTH, Quattro/OSM, Quattro

6c68d446a05cd312f9a4cc4e2950fe45fef5cdf8 authored almost 8 years ago by Al <[email protected]>