github.com/mwmbl/mwmbl commits | Ecosyste.ms: OpenCollective

Merge branch 'master' into analyse-links

26e90c6e5791d3636cb72ea103f88ad39cf74a06 authored almost 3 years ago

Merge pull request #38 from mwmbl/stop-indexing-partial-words

Improve handling of partial words

07d4b36052ce664d56a0e03695fe24a969eb1088 authored almost 3 years ago

Analyse links to find most popular ones

d77b72d7dfb13ad36c00446357606bc1fd79db22 authored almost 3 years ago

Improve handling of incomplete words:

- Correctly generate regex for incomplete vs complete words
- Return more than one top word fro...

fe6ace93e6efa55adfae19261a5308a3e239a903 authored almost 3 years ago

Use python 3.10; complete terms

7d829bc3197556cddd46214a804d59f7f21fdff8 authored almost 3 years ago

WIP: implement term completer

3c75dd1a74ffc86308d7ffe98abe9ec1afeee217 authored almost 3 years ago

Don't index partial words

01a21337a949af5d4e88c21534212f53e2e4c451 authored almost 3 years ago

Remove some debug print statements

2ef8304919a556de34e4cd590eeb62e009302a02 authored almost 3 years ago

Merge pull request #37 from mwmbl/index-mwmbl-crawl

Index mwmbl crawl

66696ad76bfe07523a7966b466103f31935036a8 authored almost 3 years ago

Index Mwmbl crawled data

5b89bbf05d2b67e9071265ae44a609e58e029d73 authored almost 3 years ago

Analyse the pages that are crawled most often

ef36513f642b0bbed3234ad5d725733cfa24c5cf authored almost 3 years ago

Analyse crawled URLs and domains

70254ae1600f42bce8974af1c047dd6ae78487b4 authored about 3 years ago

Add script to export top domains

171fa645d2f8dc00eec452859357e514eb9fb018 authored about 3 years ago

Merge pull request #36 from ColinEspinas/remove-old-frontend

Remove old front-end files and routes

908a9cf0b6bb4f5bb31dac9d18ddeb152cd06e30 authored about 3 years ago

Removed old front-end files and routes

3481ad372bc5e4dba8c6f71bd1a0580ebb28d6c5 authored about 3 years ago

Add CORS; revert back to previous index as it timed out deploying

a41088ca9ad7fdcac952a3be84c086316d9e77fa authored about 3 years ago

Export URLs to sqlite for evaluation purposes

25918e42ef7fbfb894acc62853e862f5c59dc443 authored about 3 years ago

Merge pull request #31 from nitred/fix-python-m-run

Using the app object to start uvicorn, instead of using a reference like "mwmbl.tinysearchengine...

ae7312c32afdea19d3fc2286c5936b58d86f7303 authored about 3 years ago

Using the app object to start uvicorn, instead of using a reference like "mwmbl.tinysearchengine.app:app"

- fixes the issue when running the server using python -m mwmbl.tinysearchengine.app

When runnin...

fbdb93c86ab17f012aa52f9d8c2a3b422846dd4b authored about 3 years ago

Add a component of the HN domain score when ranking

e6655101ef53fadad865374ca00cb629838fceb5 authored about 3 years ago

Add .gcloudignore file to fix gcloud run deploy

f347fe29ac7e1798702b2b643a99c2f0827ffa00 authored about 3 years ago

Explain pronounciation

3f74229ae993cc06a2c2fe1c90ad8f531bace540 authored about 3 years ago

Merge pull request #25 from ColinEspinas/search-debounce

Added debounce on search input

02bcef640cfdc48f561e09f54e3ec2634b2e19fa authored about 3 years ago

Merge pull request #24 from nitred/config-and-entrypoint

added config and binary/entrypoint for mwmbl.tinysearchengine

3d7e655ebceacf92e380ab98f8410c93c33324ec authored about 3 years ago

Added debounce on search input (#8)

c636be90898f5be8587fadfe65825d287bde25a1 authored about 3 years ago

added config and binary/entrypoint for mwmbl.tinysearchengine

- using pydantic to validate the config
- added a default bootstrap config at config/tinysearchen...

a72a08a7d9cb44213c6ef2c5c203a912cf64bc36 authored about 3 years ago

Merge pull request #18 from nitred/mwmbl-package

renamed package to mwmbl

da8797f5ef7b8ec1e6fd51edbcda47228bc13b70 authored about 3 years ago

Merge pull request #21 from ArcoMul/add-dev-instructions-to-readme

Add development instructions to README + fix .gitignore

0b7bc90a058dc6df7cbe1b5a74340458c565867e authored about 3 years ago

Update .gitignore: fix ignoroing data folder in root of repository

b6c1630953f7bf03a2a0ae9edddaad05f9f77974 authored about 3 years ago

Update README: add development instructions

d5a612aa4769925b928eb905db2fd349554e7ac3 authored about 3 years ago

Merge branch 'master' into mwmbl-package

be40a15b2746b8b20acfa2a1a613fd1b47b82802 authored about 3 years ago

Merge pull request #17 from nitred/python-gitignore

added standard .gitignore template for python from the github/gitignore repo

03ca368b2acb1a23edb839df7884452d7c26f81d authored about 3 years ago

Merge pull request #13 from nitred/indexer-dependencies-as-extra

Fixes #12, Added dependencies for indexer as extra or extra_requires

0baed3780d8648ed94391c0f7501c7b1f9a6627f authored about 3 years ago

Merge pull request #11 from ArcoMul/fix-mobile-layout

Make page responsive for mobile devices

04d7cbdfe30a7365f1668901a78b9532f502024c authored about 3 years ago

renamed package to mwmbl

- renamed package to mwmbl in pyproject.toml
- tinysearchengine and indexer modules have been mov...

11eedcde8428f39e5da3becf4f8a5fa3e62a4d71 authored about 3 years ago

added standard .gitignore template for python from the github/gitignore repo

91b357b6e29e27f36286593fce345b788b4c24fa authored about 3 years ago

Fixes #12, Added dependencies for indexer as extra or extra_requires

- dependencies for indexer can be installed using "pip install .[indexer]" or "poetry install -E ...

c02c0522813b0cc836eccb962bec113242001ee5 authored about 3 years ago

Decrease font-size of url so that the title stands out more

e773ff68e5952a0f799cd3a69b42876bf45fa5c5 authored about 3 years ago

Make page responsive for mobile devices

4e41f68a469996471a97448e634fbacac772bc76 authored about 3 years ago

Merge pull request #6 from ndren/master

Do not send Referer

acb2d19470a69a1be6b3dc6f34d493138f5ee54a authored about 3 years ago

Do not send Referer

389d0abcc1f48f01dae488338ee01ed1a6f21990 authored about 3 years ago

Merge pull request #5 from ndren/master

Add Matrix button

61e5dba20dcd4b0e6aa3c72dc629fb88238b4cb3 authored about 3 years ago

GPLv3 -> AGPLv3

a09340891aadaa3a4f47663791a6179f0d171c30 authored about 3 years ago

Add Matrix button

0cd2bd5346ad928fc8c51bde4edb8a024c77c9ab authored about 3 years ago

Merge branch 'master' of github.com:mwmbl/mwmbl

0ea7a0c031485a1105e5326511b93c41810d663e authored about 3 years ago

Add link to Matrix chat

b47de434ddf2f7eef32306f895492b23eb1ae32a authored about 3 years ago

Create LICENSE

8fcc75f037b307cbd9ad97ff17cdfc3203a9fb90 authored about 3 years ago

Move indexer code to a separate package

baede32298b712828ef13ba13ae115903a3aa321 authored about 3 years ago

Remove debug print code

8cfb8b7a44bfbe7dda33c25467b473c918b59bea authored about 3 years ago

Update domain name

794af00bfb454f4bf20cac4ece3b2aa5dc1b7665 authored about 3 years ago

Write readme

722328efa52271c2ca158dfc51f49e5da2975bbb authored about 3 years ago

Look for onchange events to get it working on mobile

6ab961d070e364061dd7ea01757d06ad9c33e98f authored about 3 years ago

Specify correct host to make the app available

0721ec0f814e24cf6d918fed773e7df7f247b63a authored about 3 years ago

Get Dockerfile working

7e520fb32ff43c7319cd8762daae2d030b517efa authored about 3 years ago

WIP: implement docker image. TODO: copy index and set the correct index path using env var

9c65bf3c8fa2e8383756be145f17f905e4898b8b authored about 3 years ago

Prevent default for up and down keys

f754b38f713bc8d0c69cad554858df6df5b92fa3 authored about 3 years ago

Improve focus on reload, back, etc

8f8fc43c9f2552fa464a46403f60b614d3acc048 authored about 3 years ago

Make Enter key work when pressing Enter

202ef35d7afe733f29c9e9fb93786c564295e368 authored about 3 years ago

Follow selected item on enter

30a00425aeda35f3ecf9cccff7193d6152b532d9 authored about 3 years ago

Select item with arrow keys

5e7c5a905e3a1b1c0d79b633a03c75cddfb8e5da authored about 3 years ago

Add background colour and hover highlighting

2d7bb0efd70d520a8ab22edaea207fe4fe8830e5 authored about 3 years ago

Improve styling

c22f522c077236f5d10a2fe7334b5f0cc50f9ac8 authored about 3 years ago

Show the URL

7c745ef87b6bad186c0d6ca14a10f520ac7a3705 authored about 3 years ago

Format extract differently

585f4bd00c0f2243b26a326bf5ca5fbd13195bc6 authored about 3 years ago

Prefer items that find the result early on

734798e4de873140a70d8121f7d587c9f51c5c0d authored about 3 years ago

Analysis to confirm that 'leek and potato soup' page was really missing

9ee6f37a608c9d3bfd042657d056334e08004afa authored about 3 years ago

Show the extract

4cbed29c08e28f8481942fb1402f04d5e047d1fb authored about 3 years ago

Index extracts

16121d2b1968d709fc30bb96db30ac6591c77c1b authored about 3 years ago

Filter results with low scores

4fa1c4a39a96b9514d80225739e5277c75ba1fab authored about 3 years ago

Improve results ordering

6b72a056b2a6165bc230f05abbf835ff59cbbcb3 authored about 3 years ago

Bold search terms in results

cc290bfc07f4e0f81ac66e8507d9922e4c2fdb89 authored about 3 years ago

Add css

e4d2a45d6c5f61a124c44e3e2845f3a4546a3d89 authored about 3 years ago

Set cursor at the end of the input

1d8b37add17e847d39745aeb8c59c983ab7a1066 authored about 3 years ago

Update results as you type

af29b4c039c5d9450b8b6d66a2edd3d649d546ad authored about 3 years ago

Add search page

23eb3418325e9efcd882449ddc1fe60874408c62 authored about 3 years ago

Add an error state

869127c6ecf3340f4d24a916f0d78fce1b836a86 authored about 3 years ago

Index common crawl data

2844c1df75dd6d83020770dbbfa411fa1e6c491c authored about 3 years ago

Add spacy

65b366d30d89c5376ee4827d5be7eb611c6f66a9 authored about 3 years ago

Run multiple processes in parallel

16a8356a23c0f0ff0a192f71a736fcc23c9420be authored about 3 years ago

Output processed items to an output queue

34dc50a6ed26be523903dc167ca8d45293f84d13 authored about 3 years ago

Use our own filesystem-based queue

c46257c6d1f2c70a635d94b5bf391addb32c6a78 authored about 3 years ago

Use multiprocessing

a76fd2d8f927171fddb563e879875f0a68d2667f authored about 3 years ago

Save results to gzip file

2d554b14e7b9a7d9acc9bb118171e469276110e3 authored about 3 years ago

Extract locally

2562a5257a4e95d0caf1640295d806d9683e59c8 authored about 3 years ago

Extract archive info

c151fe377766b06c3489817539bd932812d27429 authored about 3 years ago

Add EMR deploy scripts

a173db319b5f62606d7b7ca7bd53ad59cd65c611 authored about 3 years ago

Optimise imports

14817d76575b48492bd13f21f23127cafac4d20e authored about 3 years ago

Add common crawl extract script and dependency management with poetry

312f32bf61951a191b2fe17771bdac28d9eb0247 authored about 3 years ago

Improve typing of indexer

896f782379b086e11e665f0bd99b1da0730c28e4 authored over 3 years ago

Limit number of chars used in query

0578f41a7343cf66c111708957addba12294989a authored over 3 years ago

Abstract index to allow storing anything

c81fc83900b0aff547e36bd49679c3c3808ab6cf authored over 3 years ago

Count terms

fb5b6ffd4511b7cf758202ec4c2dbe59dcb31888 authored over 3 years ago

Optimise imports

62d22d9d52be4f6e3f8618d5368a7515fd920658 authored over 3 years ago

Replace dots in query with spaces

16aec145d04133541c7e8fac69c2bf67707e9402 authored over 3 years ago

Check for term in title

550c6f6acca8edf83c0b66425ea4c65429ec6221 authored over 3 years ago

Order results by Levenshtein distance to improve recall

d6cc81278f705e79161aaef9a3a5ad94d7b93a27 authored over 3 years ago

Use top urls for performance test

0e3069fdb36a8762a2e64c545d8d1e8e4f43d69e authored over 3 years ago

Index queued items

974f18647a7479c5f2535f3aadef67e68ed58918 authored over 3 years ago

Smaller queue

87fd458218cf9566c608b2f5e2954f6c396c4d4c authored over 3 years ago

Use a filesystem-based queue

cc841c8b7ee23bb866002ca17f5b68501039b629 authored over 3 years ago