github.com/ScoopInstaller/scoopinstaller.github.io-indexer commits

Merge pull request #19 from ScoopInstaller/multiple-bucketsources

Support for multiple buckets sources

9c1959512f1143580908be55ae299413069dd68e authored over 1 year ago

Add tests for ScoopSearchIndexerTests

df4e9d48c26dfa03ef54323bf160a394daabe4ce authored over 1 year ago

Add coverage for BucketsProvider/BucketsSource

3a794cd701c746e065f85e0c0dde36317c763297 authored over 1 year ago

Improve coverage on Manifests

083278a513b1ac4ff79317a5750dc18d14dc7732 authored over 1 year ago

Add a test to check repository redirection

ed3441d3e9017bf2c166bdbf4f175a26f6845a91 authored over 1 year ago

Replace CancellationToken.None with new instances

4f5ef7bf3d4e29e5cf029fc905878df811ef6daa authored over 1 year ago

Update unit tests + remove single quotes in logging

9edd943a613fb369aa722ca2aea7540c5e8ab484 authored over 1 year ago

Deduplicate buckets in ScoopSearchIndexer by comparing their Url and ignoring the casing

b2b4f9f940e58617655bdcb48cfe8d1c4efd60ad authored over 1 year ago

Fix a bunch of tests and ignore remaining files

704783ee3e4801b78828b09dc14afa28e2658b24 authored over 1 year ago

Massive refactoring to use IAsyncEnumerable and to support multiple providers (GitLab...)

1567d40ad8b0a46f00076b47d25761668c67a8d4 authored over 1 year ago

Overcome 100_000 results limitation when querying all the manifests from the index

371b100b1bcb8ed52d6fdee5cd177efc32705a4b authored over 1 year ago

Update README.md [skip ci]

bfb6496d44f5509ddba2e46c0951904c4bdcea5e authored over 1 year ago

Merge pull request #18 from ScoopInstaller/no-exclusions-list

Remove exclusions list to index all buckets, even the meta-buckets

f3db0c294d80f5db34494abab0c51572cd0c20a4 authored over 1 year ago

Never flag an official manifest as a duplicate

6114c770339160ee6160184980c0b160d50dbbfa authored over 1 year ago

Remove exclusions list and custom HttpClient used to not follow redirections

5d1c6673696b30e64a066acc9e412728a2536b00 authored over 1 year ago

Run indexer every 2 hours [skip ci]

18ec2bf84b30b18d67d232df6bd75607a565fb6f authored over 1 year ago

Use variables/secrets from configured environments and add concurrency check

fca70b7ab0fc828a46ef17bdf658cecc5a536d49 authored over 1 year ago

Use upper-case env variables [skip ci]

8b9eb0a7a8c3583e04d4c005f0aeb50e040ff155 authored over 1 year ago

Indexer selection based on branch

a70a5dd1e1b4b1abbd67c4dcba43e97d9a4b057d authored over 1 year ago

Use auto-generated GitHubToken instead of PAT

12d8ecf95b377c8e91278cf46093dc7ba7b5cedf authored over 1 year ago

Use variables instead of secrets for serviceUrl/IndexName

1f8721ae3a4a0ab56c516f1b75d1f9dc80b4055b authored over 1 year ago

Move GitHub buckets search in GitHubClient and search for buckets availability concurrently

7f8b524dd0f91b04f0c6cbe7a09c835f228a33e9 authored over 1 year ago

Remove obsolete custom header

b1c8c86b524ea2c2f6aa82a40b67067abcc33741 authored over 1 year ago

Remove useless this. modifiers

746a426752dc73e69787e01296780cb32f92853d authored over 1 year ago

Remove unused GitHubToken

63326756c78c3b0bea5130f25a7c634e6bf1acda authored over 1 year ago

Run indexer only twice a day

e0316f6623a77d9482f2829941437a86b6d02f41 authored almost 2 years ago

Fix a bunch of code smells and warnings

880e213ae71cbe972650b623610f99d6395f4d35 authored almost 2 years ago

Update Sonar key

fd3646254838089cc3548a85598c7e3a54269d92 authored almost 2 years ago

Remove useless GitVersion.yml and remove verbosity=detailed

2b7f953d942b0da78a023d7fd29d3a997cc0f933 authored almost 2 years ago

Update README

1efb34e3adc8fa47160b8411c14795b666a24a9d authored almost 2 years ago

Increase max duration for GetCommitsCache test

2d415302775f1b99d430aa0077c22d6d66424b1c authored almost 2 years ago

Allow null version for duplicates detection

c75591959004819a0487cbaafd200bbba94f389c authored almost 2 years ago

Normalize (trim+tolower) values used to identify unique manifests

451f10d47db55c9652fbe005916da765a6594c81 authored almost 2 years ago

Use only Name and Version to detect duplicates

64715dad33a407813bffcf07e0c90e922767c2f1 authored almost 2 years ago

Batch documents updates to avoid any timeout/update limit

e45513742cdc79c261f48a86a1b1f60a169800b0 authored almost 2 years ago

Generate hash from key properties to detect duplicates

524b6464d226b2ed41d358fe222927c8accc5d68 authored almost 2 years ago

exclude files starting with a dot from the valid manifests

7d228c4cd8042f154c0651d7dc66534e0ee9c304 authored almost 2 years ago

Remove author and name from the index (not used)

4023b13ded332c4a58c9a5de5090980d7b27af95 authored almost 2 years ago

Properly handle duplicates with deterministic order and exact comparison

3d04fbd1a29f67e1f0145f8f19d5d1c042cbfa0c authored almost 2 years ago

Ignore ManifestHash property to exclude it from the index

62d4f9e73e6902586fc6aceb14d7dad7345d0a94 authored almost 2 years ago

Run indexer every 2 hours

ff73314d5e0f4cb6f88965e1b1d0a083986314be authored almost 2 years ago

Merge pull request #17 from ScoopInstaller/console-app

Transform to a Console application and add duplicates detection

2a5997d1e059708bc2c860fbcfcef533b16017bb authored almost 2 years ago

Add GitHub token secret in tests.yml

b60833c0fee5d2ea119a93785ab10a5589e512ba authored almost 2 years ago

Update GitHub actions

cbba1b6a87dea82e924eaacbef21e53977cd1697 authored almost 2 years ago

Detect and mark duplicated manifests

12177b12260039ae3ca4126a54dfff81a993ed0d authored almost 2 years ago

Use IAsyncEnumerable in ISearchClient

27e60fe63a05f57ef7265e0153ae8c857c3afa93 authored almost 2 years ago

Remove duplicated when using different casing

e63946eb94a7029b424ebd27aae605db7349614c authored almost 2 years ago

Update logging to not use string interpolation

5c681c23d74d745b60e3c0237ad2036c8aa3e7b8 authored almost 2 years ago

Add thread id in logs

1ea0cc93f0d42e148cc5de42a483d03da4785df6 authored almost 2 years ago

Update unit tests accordingly

3b6ecbae2c36a2c14bb00d941481b5647aa77986 authored almost 2 years ago

Refactor the project as a console application.

0d12e4b20a40e8bc3ec527e9f5002e2044fedb0b authored almost 2 years ago

Add ScoopSearch.Indexer.Console project

2e679a2616fcd786f4f047586bcda519963d6613 authored almost 2 years ago

Update namespaces and use file-scoped namespaces

23b66c4b729a7cffea41a300e4e5369446d7c78e authored almost 2 years ago

Add Directory.Build.props

bc909981cf992d39ee29d766794f39136089c94c authored almost 2 years ago

Rename ScoopSearch.Functions projects to ScoopSearch.Indexer

080e1d5207567d68b45df8e84beff37d8d7b47a8 authored almost 2 years ago

Merge pull request #16 from StarsbySea/update-urls

Update urls

31f40828d6de8c0608b0b43babd54f3cfc9d46f9 authored almost 2 years ago

Update GitHubClientTests.cs

b6e85c0a613a91743e1b9cd67222078891ae3170 authored almost 2 years ago

Update settings.json

3ace27d8be4e6dc3ddb1e0c6e26d6b7e2cb32106 authored almost 2 years ago

Add OperationId as a second sort to properly sort functions that have the exact same FirstTImeGenerated

60aab134cd0b9b96fea340bbbffc61adbb6dc529 authored almost 2 years ago

Rewrite IndexingLog function to take full advantage of KQL language and to always return the full log of the last indexation, properly sorted per bucket and timestamp

55ccc08499ae4bd003adcdd2868c67c0273d08ed authored almost 2 years ago

Enable sampling with excluded types used for logging

f52c20b934acd67ec06f997ac72ff07d3924340e authored almost 2 years ago

Improve logging verbosity in DispatchBucketsCrawler

8e02a40950bb7ec4b7d80cdf6689d562152c3ba7 authored almost 2 years ago

Add OfficialRepositoryNumber for unit tests

d595521d73bcc58c99e228e38d7a9167a2d540db authored almost 2 years ago

Fix inverted OfficialRepositoryNumber property and add this property for manifests comparison

1b2464446c10a227fadd8335ae4255ee822fb2e1 authored almost 2 years ago

Merge pull request #13 from ScoopInstaller/monitor-logs

IndexingLog function added + reduce logs ingestion costs

a4c843be8d96940bfe71f64f4b600869af6e18cf authored almost 2 years ago

Add IndexingLog function + reduce logging for dependencies and perf counters

e95e9a1e9900615e32f84fb65d62f2c37cad0207 authored almost 2 years ago

Add AzureLogsMonitorOptions and slightly rework options configuration

40667b0d8ea7c3cfb082fe1a0276463e1b1ebccc authored almost 2 years ago

Merge pull request #12 from ScoopInstaller/nuget-latest

Update to latest Nuget packages + Update to Azure.Search.Documents

4650b84650f0471de03dee32a41b8acd403bd637 authored almost 2 years ago

Properly detect bucket/ subfolder and fetch only manifests in this folder

4819081f832b27b7368a15c951ec7d99f7acc0b0 authored almost 2 years ago

Improve Json parsing of License and Description (because of the switch from Newtonsoft.Json to System.Text.Json)

ede9a6f31a66cae4750eec0a2acf48952d3d93b1 authored almost 2 years ago

Downgrade Scrutor to 4.1.0 because of https://github.com/khellang/Scrutor/issues/171

936ada3fc0059dd7cf09d5b426bb8cdb2d9fa27c authored almost 2 years ago

Simplify options management by using ConfigureAppConfiguration() and by removing IOptions<ExecutionContextOptions> usage.

600bc1afdcf21983abe320737d43108e6a480d98 authored almost 2 years ago

Replace Microsoft.Azure.Search with Microsoft.Search.Documents and NewtonSoft.Json with System.Text.Json

bb491d3b7983e3fa9e74a044ef4ef7ee040f1101 authored almost 2 years ago

Update to latest Nuget packages and fix nullability checks

16f2214590257c0d4ef6d9ce1884761c8be97802 authored almost 2 years ago

Merge pull request #11 from ScoopInstaller/sonarcloud-fixes

Fix a couple of violations reported by SonarCloud

cdad93062b63ec4e62e4f2d458de6546d475b216 authored almost 2 years ago

Fix a couple of violations reported by SonarCloud

5d8ea954be2d62daf8f5e6ebc0f891bb6ebba0fe authored almost 2 years ago

Merge pull request #10 from ScoopInstaller/sonarcloud

SonarCloud integration

cbb0dcdfdfb3d9953d93a5bfb5f10d6a567f97d9 authored almost 2 years ago

Add badges

7357d4c1cccd9acfdb77e151af306d030e39f8c4 authored almost 2 years ago

Use vscoveragexml instead of opencover to properly report coverage

d35084a3c8421b41a4b9ab91b38e7d6bb600fa59 authored almost 2 years ago

Disable shallow clone for Sonar

8feb8656a063da6f072e8cf2e8d6a9e79e083bb0 authored almost 2 years ago

Fix tools installation

7c7a32f20497cceeba165745e09d06f2c951d130 authored almost 2 years ago

Use dotnet-coverage to generate coverage.xml file used by Sonar

0470a4c803468a2d9dc4396c1727723d10ea7114 authored almost 2 years ago

Update commands to linux format

518091345abfe82784ef7907bc6e18855ae95438 authored almost 2 years ago

Send coverage to SonarCloud

9d0d9f61879fb3c90483b36fd871ac3581020e4c authored almost 2 years ago

Add workflow_dispatch trigger to tests.yml

7ac00e7acebab81200464ce2a55f67e62303ea95 authored almost 2 years ago

Split GH actions into 2 workflows (one for tests and one for deployment). [skip ci]

105bc4e0975116af623dd282754ed2bc7adcf36e authored almost 2 years ago

Don't deploy GitWindowsMinimal with the Azure function as git is already available on the hosts

8bdaf93d92be179603b676478681e019bd0682be authored almost 2 years ago

Change filesystem attributes to avoid an UnauthorizedAccessException during Azure functions execution [skip ci]

d70de26e3505a780a168a1294bc18a140095e133 authored almost 2 years ago

Fix flaky test + properly display git version without newline [skip ci]

ed55a4370a174db71abfbbec205be8b348c50892 authored almost 2 years ago

Use Git from path by default, even on Windows

986e610d003fb0b4941550a3f43f5cb632bc6305 authored almost 2 years ago

Fix deployment conditions checks

4063122851a2d370e12d2b24d7aa0bd68020b943 authored almost 2 years ago

Merge pull request #9 from ScoopInstaller/tests

Add tests coverage

79450305cfcb51ec6e624a454f9af22fdbd1eee1 authored almost 2 years ago

Add Host startup logging

5e6aec5fb3d3d227663e55cf7bd565e85b7f6c9e authored almost 2 years ago

Use latest libgit2sharp preview for Unix/Mac compatibility

28bce01eb196649921bfc7af35f2bdba7cd2a2d0 authored almost 2 years ago

Update GitHub action to add tests execution, to run the tests daily and to deploy only when running a manual workflow

d69cd762e0b08f0c986499b222a9093de055a738 authored almost 2 years ago

Add tests for DispatchBucketsCrawler

fb95ba76e7f05ad9b50c13522aea7957e8cf3ba5 authored almost 2 years ago

Extend LoggerMock with customizable TImes

b68cb3e4563e156e59b6c8f7ee157fbb7434dbeb authored almost 2 years ago

Add tests for BucketCrawler

1e8fcded5c4ad11eca2ef42564119cff1e9f361c authored almost 2 years ago

Keep only one if statement to detect if there is a valid branch on the repository

0c5425202c1ed611d9f820baf41fd4e82350f44e authored almost 2 years ago

Add UT for GitRepository.Dispose method

810d99939d3137740ff7e9c621892d104db15df8 authored almost 2 years ago

Ecosyste.ms: OpenCollective

github.com/ScoopInstaller/scoopinstaller.github.io-indexer