Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/censusreporter/census-postgres-scripts

Scripts used to set up census-postgres on an Amazon EC2 instance.
https://github.com/censusreporter/census-postgres-scripts

it's aitsn, not aits

04e068e11b1a76294004104afba73e3185c13eb9 authored about 7 years ago by Ian Dees <[email protected]>
Rename the index tiger file to .sql

6e535fcb34436ce81120d8a07eae119b586987ae authored about 7 years ago by Ian Dees <[email protected]>
Use correct table name

42c436dc8f9dea4050e89c21438b93251b1ee957 authored about 7 years ago by Ian Dees <[email protected]>
Don't do anything with tabblock because it's not in ACS

30a25444287d751b434f11c41ef12e5cc8e8af22 authored about 7 years ago by Ian Dees <[email protected]>
Sort the TIGER download types and remove blocks because ACS doesn't support it

a763a8d2f702899895e77dbb0b7a452936f9c84a authored about 7 years ago by Ian Dees <[email protected]>
Ignore robots.txt and use HTTP to download tiger2016

8f8a13917ebb5c93337af9e6af4e6aaa91f2a689 authored about 7 years ago by Ian Dees <[email protected]>
Explicit directory for tiger2016 download

97c812d7c7633f5c08bc9ee80215a03375da3e1b authored about 7 years ago by Ian Dees <[email protected]>
Tiger 2016, not 2015

d1b008b4c1bea3a0d60fd1ef1d80d5bd883b2743 authored about 7 years ago by Ian Dees <[email protected]>
Geoheader files moved, too

9df7d3ad03b3865ffcd3642ba9cc8001225266cd authored about 7 years ago by Ian Dees <[email protected]>
Point to the right 2016 5yr data files

568fabecbe706e279b06e090fe667c7d2b2b2b4f authored about 7 years ago by Ian Dees <[email protected]>
Adding the 2016 5yr scripts

d35553781f7e1ed8660326f98041472b8ae9d85c authored about 7 years ago by Ian Dees <[email protected]>
Add import scripts for 2016 1yr + TIGER 2016

0a737b3df04b472c74c682ff97d50f662864d035 authored over 7 years ago by Ian Dees <[email protected]>
Census stopped packaging the 5yr files deep in a directory hierarchy

e4bddb68923a01769297973b6d34651bfa55cc1e authored about 8 years ago by Ian Dees <[email protected]>
Prep scripts for downloading/importing 2015 5yr

df7ad6e7ba4067b49091008cfc63f3c0a62cd73b authored about 8 years ago by Ian Dees <[email protected]>
Updated the file path for acs2009_5yr

Corrected the extraction behavior to accurately extract at the correct location.
This updated ...

e79e659427ac358d6e7360ac449ad40f9979f640 authored about 8 years ago by Peter Wang <[email protected]>
Update README.md

86f0588b7f86dcce0b8031089c9543988d3350ea authored over 8 years ago by Ian Dees <[email protected]>
There's no acs2015_5yr yet

1c9b8f07bbc4e062ecb40d9a9896fd04c58f9689 authored over 8 years ago by Ian Dees <[email protected]>
Pipe the shp2pgsql output straight to psql to save disk space

3341797a1aee64c8bf5dfe85d0e6f2405c2fe546 authored over 8 years ago by Ian Dees <[email protected]>
Cascade the schema drop

544e2187f248d136d682c8bd452057cdfe9848a5 authored over 8 years ago by Ian Dees <[email protected]>
Work on one kind of TIGER data at a time, cleaning up as you go

8cfdd847f29bc8a10849537bf01db860e69b5f3c authored over 8 years ago by Ian Dees <[email protected]>
Clean up the 2015 tiger import script

e5fed8509582397da24328716a69aa86a438f0e6 authored over 8 years ago by Ian Dees <[email protected]>
Add the TIGER 2015 download, import, and index scripts

d80b1ab87d2c9710b895d48bcb682822ec9e3f61 authored over 8 years ago by Ian Dees <[email protected]>
Add a rename step for the post-download step of tabulation metadata

085d4527714620c61ac059c24cceb95c36538f57 authored over 8 years ago by Ian Dees <[email protected]>
Update download scripts to include installing aria2

58dd56b7a2cba7b1f5689c196305ffdb2c7a7cbe authored over 8 years ago by Ian Dees <[email protected]>
Add 2015 1yr scripts for downloading

f4cf0aaf1d99fe7a49db8477373c16d90675f6ba authored over 8 years ago by Ian Dees <[email protected]>
Initial commit

This script creates pseudo-tiger tables for the different aiannh
sumlevels (250, 252, 254). Thes...

6f9844a7039a17c743153bb26d63bea05a796027 authored over 8 years ago by Tushar Chandra <[email protected]>
Initial commit

Create tiger-style tables for aiannh profiles with sumlevels 250,
252, and 254. Insert rows from...

9b4bbfa4c974abe9e64b6ec9c14b6792255d1a6a authored over 8 years ago by Tushar Chandra <[email protected]>
index sumlevel on census_name_lookup

56d1371595e47d0f94d9de76e4fa6a5905d37acf authored over 8 years ago by Joe Germuska <[email protected]>
Merge branch 'master' of github.com:censusreporter/census-postgres-scripts

3243f783ffdf992d050106d03a30c3cab75f414f authored over 8 years ago by Joe Germuska <[email protected]>
Switch to PGHOST instead of sudo for 2013 tiger import script

e08b859622dc452f6be2fb2d62def7d07ef447f0 authored about 9 years ago by Ian Dees <[email protected]>
Remove a sudo

50457fbd3e7e2b89b1cfd63a73deeb6495e9ded5 authored about 9 years ago by Ian Dees <[email protected]>
Don't sudo for 2014 1yr import process

cf4dc7aa14166dafcb4a23f08ccf75640dd408c8 authored about 9 years ago by Ian Dees <[email protected]>
acs2014_5yr has 121 sequences, not 165.

76b4e3a544fc6583cd5b46a703e24c00472edd81 authored about 9 years ago by Ian Dees <[email protected]>
2013 3yr has 169 sequences, not 165

f9816710eaa8d389323ff39e81e4cf8f9f79f832 authored about 9 years ago by Ian Dees <[email protected]>
Update some of the postgres import scripts to support remote hosts (e.g. Amazon RDS)

462e85f6d84bcf9ca5426343f4cbfa250eb0405e authored about 9 years ago by Ian Dees <[email protected]>
Add a step to move the table lookup file to the place the next step expects it to be

1a2c3d3c5e28dc4c2eed8d0ff2263ced3b7c871b authored about 9 years ago by Ian Dees <[email protected]>
Add the 2014 geography files too

91bae9e8b8087501b69001f6b05166680e7044da authored about 9 years ago by Ian Dees <[email protected]>
Add 2014 tiger download/import/index as well

b280b58af9f4cde4273dfe3439664c23897da778 authored about 9 years ago by Ian Dees <[email protected]>
Add 2014 5yr download/import scripts

0399f46de928e992c0277e571dbaeab636e9370c authored about 9 years ago by Ian Dees <[email protected]>
Merge pull request #10 from talos/talos-fix-readme

Fix request-spot-instance aws request

f0286ddd657b4eb849c838eeb6c08442884e5710 authored about 9 years ago by Ian Dees <[email protected]>
Fix request-spot-instance aws request

45b03c909e9fb328a53392d58172e159b53e04be authored about 9 years ago by john krauss <[email protected]>
Download and import scripts for 2014 1yr

22b33c4b24c6de9357a5472063912ff171c71888 authored over 9 years ago by Ian Dees <[email protected]>
Merge branch 'master' of github.com:censusreporter/census-postgres-scripts

6c63d8ef6b7aa86daab49262c8919eecd9ed2bf9 authored almost 10 years ago by Joe Germuska <[email protected]>
Add the tiger 2013 indexing SQL

5a8d8395ef3c125e44823e9840983de295d9c9ae authored about 10 years ago by Ian Dees <[email protected]>
Add tiger 2013 release scripts

ae8837082d19a51da08629739f144e8d3ce45408 authored about 10 years ago by Ian Dees <[email protected]>
Move the 2013 5yr geo files out of their subdir like all the others

8ba2d69a7fa2fca27575658d4c8641a5965abfd0 authored about 10 years ago by Ian Dees <[email protected]>
Don't preallocate disk space

f0c86008210854de484779dc1a15d477b388357c authored about 10 years ago by Ian Dees <[email protected]>
Add 2013 5yr download scripts

d3237ebae37fcab7f066f7d8fc62270903c7ac75 authored about 10 years ago by Ian Dees <[email protected]>
Correct 2013 3yr URL

62e7265984f0c33d3ad4fe071f3513ee6e04d4a8 authored about 10 years ago by Ian Dees <[email protected]>
Add 2013 3yr scripts

17d9a3100c6b737701d04f4d99289e186b985b5e authored about 10 years ago by Ian Dees <[email protected]>
Adding 2013 1yr scripts.

140e225e8c11900cf47a079a65f9b97e94196cd7 authored over 10 years ago by Ian Dees <[email protected]>
drop if exists so that this can be rerun on a non-fresh schema

fecd74c51ae890b568752ecca028e3db0bcdd3f6 authored over 10 years ago by Joe Germuska <[email protected]>
be explicit about geom column name. some versions of shp2pgsql use different default names

90848e0ff00a5971b465def7e9179d26b39a786c authored over 10 years ago by Joe Germuska <[email protected]>
legislative districts redrawn, ACS 2012 data is aligned with new districts. ref #5. This may be all that's necessary for new loads

63cf87bc0295c3f4f50a0d458343a09d6f129ae0 authored over 10 years ago by Joe Germuska <[email protected]>
remove special handling for NYC. Closes #4

40755fa2b915d57c36e6268d76c000c76b972f44 authored over 10 years ago by Joe Germuska <[email protected]>
remove superfluous sudo. closes #1. Thanks @pnorman

7c44ba2b26f2d85bf79070119c48a6f6841ce325 authored over 10 years ago by Joe Germuska <[email protected]>
don't load tracts, block groups, PUMAs, UNSDs for 'island areas' -- pivotal 72616990

8705b24d7f7ff53f773e77876c1eec652a2ac279 authored over 10 years ago by Joe Germuska <[email protected]>
Add NECTAs in States containment index.

457c100a6c2b64fb89461067ff357f76b12efbe2 authored almost 11 years ago by Ian Dees <[email protected]>
Add CSA inside States containment.

8e4131f873de18e803241ddbeff1a01802bf45d1 authored almost 11 years ago by Ian Dees <[email protected]>
Grant select on the geography_columns table for the OGR stuff.

17a7295c17a3fc300e012da01c5c84fe4591f45e authored almost 11 years ago by Ian Dees <[email protected]>
Add CBSA inside CSA containment.

265e839072eb240671d29a38aa017b9e10073cc6 authored almost 11 years ago by Ian Dees <[email protected]>
Remove the unneeded index drops and add in containment checks.

2e01a21247f0f63f0026cd84ab2a26f1482a818b authored almost 11 years ago by Ian Dees <[email protected]>
Remove the census_names view since it's been completely replaced by census_name_lookup

0474d0e93ac77a6bfe70831c50755974f21f5b1d authored about 11 years ago by Ian Dees <[email protected]>
Add regions and divisions along with SQL for national pop, aland, awater.

07a0707404619657c0baeb66e81ccc7b612e20b2 authored about 11 years ago by Ian Dees <[email protected]>
Use the 2012 5yr for population estimates instead of 2011.

7b85bc1b51ff3ce74e3b59f30919dac1f87b8f10 authored about 11 years ago by Ian Dees <[email protected]>
Fix up readme.

a196b07033eca0d9d1701dda38dbbc7574f4f504 authored about 11 years ago by Ian Dees <[email protected]>
Put the -q for unzip in the right spot.

fe242254748e0ffff5f9bff9f1636e323dbc7c2e authored about 11 years ago by Ian Dees <[email protected]>
Run shp2pgsql imports on census database.

2c4e908ceff29ff3da4aa3340e6129a76cf6509d authored about 11 years ago by Ian Dees <[email protected]>
Add an aws CLI tool command and a start for what to do on new releases.

5cb116e6568c52bebd4b8800a7461fa5d13ee5b7 authored about 11 years ago by Ian Dees <[email protected]>
Switching back to sudo postgres for the files that use COPY.

72cad69b70e8aab696f6f494e47f345c913b1d43 authored about 11 years ago by Ian Dees <[email protected]>
Import using the census user and database.

9da8e0a84cc52b3f01db2ead974ee68d51fd31e4 authored about 11 years ago by Ian Dees <[email protected]>
Create a .pgpass file to make import easier.

32ec03ed29eb59e797afb2873660fe802012532f authored about 11 years ago by Ian Dees <[email protected]>
Trying to correct the owner of all the data imports.

9546a84f47cd62c8b273af0485fa27523ad03dc7 authored about 11 years ago by Ian Dees <[email protected]>
Update readme with the complete steps that I went through to recreate the database today.

7edac97910a9f17427c3bd1cac7caba61156f07d authored about 11 years ago by Ian Dees <[email protected]>
Trying to use aria to download data faster.

19a88a77d4f3bc0b2f95a48acb52e911aaeccbce authored about 11 years ago by Ian Dees <[email protected]>
Add 2012 5yr scripts.

90a7e696b3ad7b57da9fbdb7fffee0120007aa02 authored about 11 years ago by Ian Dees <[email protected]>
Correct 2012 3yr URL.

524a270adef1729c0ecdd53ec9e1d5211972d500 authored about 11 years ago by Ian Dees <[email protected]>
Adding 2012 3yr steps.

ef46fe2870e5a3bc8edb2e15b744d91ce9a29dde authored about 11 years ago by Ian Dees <[email protected]>
Put places before counties. Add estimates for United States.

b9f883fb6adf686fe6152ffd8caebaf7b385ac53 authored about 11 years ago by Ian Dees <[email protected]>
Fixing some typos and allowing for large aland/awater numbers.

e0ead6bcd84926a082f9ccc113212d5e49dd6a75 authored about 11 years ago by Ian Dees <[email protected]>
Add aland and awater because it might be used at some point.

6cd6c9094c960a26e802744196f182e3d457338c authored about 11 years ago by Ian Dees <[email protected]>
Add a simple name for the lookup table.

ef76f96f105dfdaa1f19796b355d1071a1bfaeb8 authored about 11 years ago by Ian Dees <[email protected]>
Add a row for New York city.

89f16b3786c1da1fa940299e3a7f20e480fb10d0 authored about 11 years ago by Ian Dees <[email protected]>
Add population to the lookup table.

c104c8a8ff85364e6e48d587060d17f969d361bd authored about 11 years ago by Ian Dees <[email protected]>
Copy all the geodata over to the lookup table so we can more easily do geo queries.

a9ae09785d8a16cc0f44e5a51c573d82a49991ed authored about 11 years ago by Ian Dees <[email protected]>
Add the name prefix index back in for now until I fix the custom table option.

1e16332d235c88af36f926bdd1ae6849927db385 authored about 11 years ago by Ian Dees <[email protected]>
Switch back to a prefix match.

500bd2d6e61d4dbe570cc137388a4642ebb68977 authored about 11 years ago by Ian Dees <[email protected]>
Reorder census_names view just a bit.

5db567deac2d14c42595648ba751d98571c030d8 authored over 11 years ago by Ian Dees <[email protected]>
Tweak the order of the census_names view.

5390125477a33c63a430b71950d0b4d7565908a8 authored over 11 years ago by Ian Dees <[email protected]>
Drop the census_names view before trying to add it.

3fa9ded4765d7d6c710f579c0e8b12dc247ca3f8 authored over 11 years ago by Ian Dees <[email protected]>
Don't index block names for now because it takes too darn long.

8c127b2d0a24fa9be9c4de6b316b5a212d887175 authored over 11 years ago by Ian Dees <[email protected]>
Move the tiger indexing stuff to a separate SQL script and execute it.

4d11f24efdb0457f59baa451e0b97485c3518848 authored over 11 years ago by Ian Dees <[email protected]>
Fix some of the tiger indexing steps.

1e766c619f217de2cac89c0f0c40af43e04c7c89 authored over 11 years ago by Ian Dees <[email protected]>
Fix ragged CSV ends and set the psql password correctly.

c1a8cefb6e1afda7f5f706c15b7fef43d114b242 authored over 11 years ago by Ian Dees <[email protected]>
Correct the tiger directory name.

802359fca61925e3e1b4efa59ead8ec2d39ce677 authored over 11 years ago by Ian Dees <[email protected]>
After expanding all this stuff, we don't need to keep it around.

656894a3023aa8528bb311b87126f98ecee60734 authored over 11 years ago by Ian Dees <[email protected]>
Make the TIGER download script more robust.

7dc71ba789be77f3323400e54d03e49b14d5dc7c authored over 11 years ago by Ian Dees <[email protected]>
Add import script for 2012 1-yr.

567c8e5d15d5dd6a4d9756cae2e729da7718564d authored over 11 years ago by Ian Dees <[email protected]>
Add 2012 1yr download script.

0e6446df06dcc9ff2edd344a7dabc30d58c30116 authored over 11 years ago by Ian Dees <[email protected]>
Parse the geoheader immediately after importing it.

12c7fbf39380fed244229b42779a0a3ce546bced authored over 11 years ago by Ian Dees <[email protected]>