Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/censusreporter/census-table-metadata

Tools for generating metadata about tables and fields in a Census release based on sequence lookup and table shell files.
https://github.com/censusreporter/census-table-metadata

Switch over to using/generating JSON docs for elasticsearch.

d52c2fb40f4a54bdfd0278947f694d318a14289b authored almost 11 years ago by Ian Dees <[email protected]>
Whitespace changes.

df81a31c0e6ead537d027a5ae41d75297ecbe901 authored almost 11 years ago by Ian Dees <[email protected]>
Add custom weight information to precomputed csv.

206401c2947b5913eefd7441b8ab345ebd952203 authored almost 11 years ago by Ian Dees <[email protected]>
Handle tables that are iterative-only better and add weight.

e64a7c71e974653b61f88adceffa541aae872563 authored almost 11 years ago by Ian Dees <[email protected]>
Don't need these indexes for now.

ded81835eb4bfb54ae3b9d8b0be7651241c75282 authored almost 11 years ago by Ian Dees <[email protected]>
Manually remove 'White Alone' stuff to make tabulation clearer.

aef58dc7fa1aa545cf87b0df078e56e01500c6b7 authored almost 11 years ago by Ian Dees <[email protected]>
Update the precomputed unified table schema CSV.

1f0424b9dc7de8d49b5f4f23a269a4af05a92f93 authored almost 11 years ago by Ian Dees <[email protected]>
Modify the schema a bit to support the changes to tabulation schema.

f93824e2bf81722480aa10e83a9061b9ac877605 authored almost 11 years ago by Ian Dees <[email protected]>
List the full table codes for each release.

Instead of using an array with single-letter codes, this lists all the tables for each tabulatio...

c1b5d788c45813345f1aa5ff3691988a25fe07a5 authored almost 11 years ago by Ian Dees <[email protected]>
Add a copy to load the unified metadata.

40f9208a3ff4ba660459eafbd2cced309a910057 authored almost 11 years ago by Ian Dees <[email protected]>
Move unified metadata CSV to the precomputed dir.

c3528f0754af34c00d085c4aba17e7c434b426ed authored almost 11 years ago by Ian Dees <[email protected]>
Move to using postgres arrays instead of characters.

6d8c094843c8b555cc8c1838ebe7a491156b3cdd authored almost 11 years ago by Ian Dees <[email protected]>
Add tabulation metadata table.

0a91eccd21e601bf75c455ab74366bf41ed5c77d authored almost 11 years ago by Ian Dees <[email protected]>
work towards a unified metadata

76e4c47068b653196036c8c9d59d9723defb22d4 authored almost 11 years ago by Joe Germuska <[email protected]>
Add precomputed 2012 5yr data.

1176885f243e050bc392f7868e904844d6c0fe23 authored about 11 years ago by Ian Dees <[email protected]>
Adding 2012 5yr metadata build steps.

7d3028f3efda77051825739d0483a6beb003975d authored about 11 years ago by Ian Dees <[email protected]>
Add 2012 3yr to the table metadata import.

f645381aba0b47e2faf0b7a11ae3cb516db499b9 authored about 11 years ago by Ian Dees <[email protected]>
Add 2012 3yr build steps.

939dd19050e3c24fb20d1a86cf323d8315e349be authored about 11 years ago by Ian Dees <[email protected]>
Add precomputed 2012 3yr stuff.

4b097997b207e64e138672d7693d695d37020f03 authored about 11 years ago by Ian Dees <[email protected]>
Improve the hierarchy and deal with wide tables better.

f7830da9b313e93a36afcc507839efae6a30f034 authored over 11 years ago by Ian Dees <[email protected]>
Keep the heirarchy around until the table_id changes, then write it out.

250256acfbfadf489b651a64ad15bb4e8a4d48c6 authored over 11 years ago by Ian Dees <[email protected]>
Drop the duplicate tables.

28a0db0ed4dab92f2d1bde14347b6e2fe86da030 authored over 11 years ago by Ian Dees <[email protected]>
Was deduping with the wrong table ID.

01fedd9a4f6bdb61686d1c5428d61f069c9b1ff0 authored over 11 years ago by Ian Dees <[email protected]>
Add/drop the 2012 1yr metadata too.

6a94fd7139f858035c1680e1c9ea172779370f91 authored over 11 years ago by Ian Dees <[email protected]>
Remove the duplicate tables from the 07 and 08 table metadata.

8677b428eb21839106e7682d4181714c1a1625eb authored over 11 years ago by Ian Dees <[email protected]>
Dedupe table IDs for the merge file.

4e779adc1139a33c9f95e288796e4edd6052bb2f authored over 11 years ago by Ian Dees <[email protected]>
Sequence number can't be part of the primary key.

df6bf7f29351366eae30c3753f4dbb1d7438901a authored over 11 years ago by Ian Dees <[email protected]>
Add 2012 1yr table metadata.

d3559f5eb2b0061133b638a3786f2258cfd3f19b authored over 11 years ago by Ian Dees <[email protected]>
Removing sequence number from the precomputed data.

88ea00c86b530aca7746c69009744a39a90108f1 authored over 11 years ago by Ian Dees <[email protected]>
Remove denominator column from median table precomputes.

f083b8d881d5fdcf190fb5eeac74d1cfb9ce912b authored over 11 years ago by Ian Dees <[email protected]>
Don't add denominator column for median tables.

63a3251dd691c0c3a704009c4ef5627ac5cc406e authored over 11 years ago by Ian Dees <[email protected]>
New precomputed stuff after previous change.

5495dfc7faef2823b61ea13ee795fe3676b2c7e0 authored over 11 years ago by Ian Dees <[email protected]>
Don't give single-column tables a denominator.

696d926b2386ea5d336c40373677577da4f38d0f authored over 11 years ago by Ian Dees <[email protected]>
Forgot the semicolon there.

ea95571ded271df24163714d40ddc9f3dd325934 authored over 11 years ago by Ian Dees <[email protected]>
Add IF EXISTS put back the topics tables back in the drop script.

c872749cfab30e7d26cacbbbbdc5c4f65b170468 authored over 11 years ago by Ian Dees <[email protected]>
Commit the precomputed CSVs with topics in text[].

1853a2768af4b8b782274203d86c1947c1802bd9 authored over 11 years ago by Ian Dees <[email protected]>
Put the topics in to text[] instead of many-to-many tables.

fab20292c7feeac6fc2970a219bcb1a49962cc4d authored over 11 years ago by Ian Dees <[email protected]>
Update the precomputed CSVs.

1f48dbd4e915534dd95e71eeb7c592d94b44d5d3 authored over 11 years ago by Ian Dees <[email protected]>
Clean up table topic mappings.

0f2203583be3f553bb2d093ef3a07e81ddb4244c authored over 11 years ago by Ian Dees <[email protected]>
Commit the new precomputed stuff after switching to slug IDs for topics.

5497850c82d444c676f16296c62244553b3c01db authored over 11 years ago by Ian Dees <[email protected]>
Ditch topic_ids and let the topic slug be the ID.

e6a0713a9f753bfbea8d6cc452f5a8c4fc530c9f authored over 11 years ago by Ian Dees <[email protected]>
Add a clean step for the makefile.

fca5e347ffec9c36e1b8d3905df2ff8ce1c8fba9 authored over 11 years ago by Ian Dees <[email protected]>
Add text pattern indexes for the table and column names.

6e2581aeeaa46bcaea056250e1a2d7a9d52c9e71 authored over 11 years ago by Ian Dees <[email protected]>
Add details about how to get the data into the database.

3fe7bb7c94db424b197e951bb39f4f6b51ed213a authored over 11 years ago by Ian Dees <[email protected]>
Add sequence number to the column_id index.

There can be multiple of the same column_id.

8c9b8ef5ddfccf915d256a28108a287b976e0d0a authored over 11 years ago by Ian Dees <[email protected]>
Add a non-unique index to the table_topics table.

7edaeda9f490fd1408f90f939b80d6098d1b2399 authored over 11 years ago by Ian Dees <[email protected]>
Ditch the foreign key stuff because we're not adding to these tables.

76c73959c2b3a5f85fca853d45a8630c4e6c3568 authored over 11 years ago by Ian Dees <[email protected]>
Add cascade to get around foreign key specifications.

a4f1ae219b2514db063d09fa314a8bcdf5511085 authored over 11 years ago by Ian Dees <[email protected]>
Fixing an extraneous character that got in there.

7e5447e25e5cf723e22d1547e3af5cc3e2e3f64b authored over 11 years ago by Ian Dees <[email protected]>
Add schema references to the foreign keys.

38a11fdec6aa8ace4c34b4472f7f01bb9e1aa2ed authored over 11 years ago by Ian Dees <[email protected]>
Add a drop script for the metadata tables.

8516c8ea7c69a820fd05704c4e21635414a58328 authored over 11 years ago by Ian Dees <[email protected]>
Format the example commands consistently.

77e672cf35d1a89024eaf0cfa7221ebedf15bbc1 authored over 11 years ago by Ian Dees <[email protected]>
Add a sentence about the precomputed data to the README.

f1b0b7ddfd76b933332b27d022dcbe84887b8d34 authored over 11 years ago by Ian Dees <[email protected]>
Commit fixed 2007 precomputed data.

0a4fa9b46a116ac1c55596a31f69c191459ce612 authored over 11 years ago by Ian Dees <[email protected]>
Fix the Makefile so files download in the right spot.

7b36c0cf31c6604602f2b9a675f5223ab2c26557 authored over 11 years ago by Ian Dees <[email protected]>
Update docs a little.

b980cccdcf078d53f1de660f5adaf3b2d81854dc authored over 11 years ago by Ian Dees <[email protected]>
Add new 'built' CSV files.

5ee0e1f214a85ac05bbf04e6f16c663376ea3226 authored over 11 years ago by Ian Dees <[email protected]>
Move the download steps to a Makefile.

cb43b785eb6aa816e942370814c3056f6322e9e8 authored over 11 years ago by Ian Dees <[email protected]>
Cleaning up older files whose functionality is now subsumed by newer stuff.

f36552229f509f4acaad3b6f2527c92ffba7ee8f authored over 11 years ago by Ian Dees <[email protected]>
Separate out the schema creation and loading steps.

afc9913079e04d6336dab879555ab2e88dda9185 authored over 11 years ago by Ian Dees <[email protected]>
Normalize the census column/table metadata tables.

8b64e14e428adf656ec6a94e0a4de657400895ee authored over 11 years ago by Ian Dees <[email protected]>
Add dsstore to gitignore.

74a2bede4ad10920ff7eabddd127e727e306420c authored over 11 years ago by Ian Dees <[email protected]>
Find the denominator column for each table.

5a0af66f1d1c376a9b1c3acc106aeee5950a6674 authored over 11 years ago by Ian Dees <[email protected]>
Compute and write out topics to separate CSV files.

cb7c7d082822d067d53cf35cbb3cbe803807df3f authored over 11 years ago by Ian Dees <[email protected]>
Handle the case where the 2008 shells use spaces for indent.

a47137e95f49c6856c2dc37b51998e44edc9b640 authored over 11 years ago by Ian Dees <[email protected]>
Titlecase the universe text.

55dc2b19964e46ebdce4b1f28a6416ddb8dc9a6f authored over 11 years ago by Ian Dees <[email protected]>
Add the simple table title.

ab527091e6bbaf680c68fc6931dcd1193bf371c8 authored over 11 years ago by Ian Dees <[email protected]>
Titlecase and other cleanups to the table title.

0228674a17d847c5d41ceb7ccf23a7b381f15841 authored over 11 years ago by Ian Dees <[email protected]>
More robust handling of universe string.

034ddfbc6d20a7f7b8bf506c0f301c372830658e authored over 11 years ago by Ian Dees <[email protected]>
Separate out the table and column metadata outputs.

d9cdec0abfc9299655af77c6e81bb5950f51427a authored over 11 years ago by Ian Dees <[email protected]>
Add a script to download the shells and merge files.

e71480c7bb86bb1be725584a64a519c04af32a81 authored over 11 years ago by Ian Dees <[email protected]>
attempt to eliminate need for 'subject area' from Census, since they are sometimes too coarse and also just aren't part of our other metadata

3dbbbee71f0b601140724f20a0a82d8a797f7bdc authored over 11 years ago by Joe Germuska <[email protected]>
integrate topic identification from taxonify.py and add lots of housing topics

86b81355091c5539267726a7d0cf9be0c762248b authored over 11 years ago by Joe Germuska <[email protected]>
clean up universe values, add some more eyeballed fixes for titlecasing and a few more colloquial replacements

b00c8c9205427573b68933c8ae10e0b32d5a087b authored over 11 years ago by Joe Germuska <[email protected]>
more work on cleaning up table names and colloquializing them

769a75515d619592c41f5a55cd9995456d53b904 authored over 11 years ago by Joe Germuska <[email protected]>
more stuff to go with the fiddling

e29dff5a73fbe103d0517abe7524218e9d66f6c2 authored over 11 years ago by Joe Germuska <[email protected]>
misc fiddling

af336fa3ddf9cb246f1b129e160e253f78220302 authored over 11 years ago by Joe Germuska <[email protected]>
s/heirarchy/hierarchy/g

ee3de546a113a05d9c63a51336f10d50c360ba75 authored over 11 years ago by Joe Germuska <[email protected]>
ignore pyc

845e88762de5de34e8a70bdea70bd192a49cb658 authored over 11 years ago by Joe Germuska <[email protected]>
Protect against trying to put the output in /

d37369b58a66802ee430218efe4e3af8dc36e9c8 authored over 11 years ago by Ian Dees <[email protected]>
Modify the README a bit to reflect my changes.

5066868d5d339fb33ad5c7ac281f388a8aaf6127 authored over 11 years ago by Ian Dees <[email protected]>
Add the table metadata schema SQL files.

ec80ab35f807dd8b0092e6ab052f9912197b4c75 authored over 11 years ago by Ian Dees <[email protected]>
Strip out the spaces in the table_id

a1eb214af8940ec582bc60fc37976b85f9cbecbd authored over 11 years ago by Ian Dees <[email protected]>
Add CSVs built from running the scripts so others don't have to.

8dc93a03062ad1b208a1d113b0ff2dec31810b30 authored over 11 years ago by Ian Dees <[email protected]>
Read shell data from another XLS.

8f964eaa156c695885660bf083fb6180919ddaab authored over 11 years ago by Ian Dees <[email protected]>
these may not add much beyond what taxonify.py does but storing them for now just in case

195d89d316dd9b67c6cf9280be2185ff52877b11 authored over 11 years ago by Joe Germuska <[email protected]>
Handle the subhead .7 and .5 rows better.

75d4e19e54d54f04f1c0d9bfe791b50ff8ffee6b authored over 11 years ago by Ian Dees <[email protected]>
Hack in a thing to read the merge file during shell processing.

2bf64242de299391b18779e4b4ce5373206121ca authored over 11 years ago by Ian Dees <[email protected]>
Strip whitespace and try to handle blank rows better.

d16e357eceaf52759ad7a222503354d47798267e authored over 11 years ago by Ian Dees <[email protected]>
Clean up imports.

08548343676804012f13e3b21f4c86000f0d46e1 authored over 11 years ago by Ian Dees <[email protected]>
Clean up the process_shells a bit.

1b9d66af562d91460988bcd8613354fda5b0fa34 authored over 11 years ago by Ian Dees <[email protected]>
Remove heirarchy information output.

8054aedf8912c406358ebc5a877019f9464183fa authored over 11 years ago by Ian Dees <[email protected]>
Add a script that tries to handle the case when the shells aren't there.

9fb0c28a3079705bbd6b40462582e61a93c5291f authored over 11 years ago by Ian Dees <[email protected]>
clarify two-word taxonify arg in readme

c2ec67f40314dde06a7951bf4381c4aedce52400 authored over 11 years ago by Ryan Pitts <[email protected]>
add taxonify script

94abc1eafef4c9115e2891fd224dead10d34c87a authored over 11 years ago by Ryan Pitts <[email protected]>
fix readme

3672a4d5515f7174fd5e540604342c4dc339f9a4 authored almost 12 years ago by Ryan Pitts <[email protected]>
initial commit

79b96c027d3926cd5921bb056f579c65a573fe1d authored almost 12 years ago by Ryan Pitts <[email protected]>