Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ironArray/Caterva2
REST and on-demand access to local/remote Blosc2 data repositories
https://github.com/ironArray/Caterva2
de2e5778c0b87ad398a60067ef179a969d959109 authored 11 months ago
5e9c0a60ceecd99a8cb345fda15f59cd7c447927 authored 11 months ago
17d34198b1adb30fcf4c01b442eb171883bfca8d authored 11 months ago
This fixes a reference to a global variable instead of self.
cb6d319fa20cbeac81e0e6e1c502a3e08bae1c9a authored 11 months ago
It happened to work because the published did have a global variable holding
that instance.
This defines some exceptions to be used in publisher root classes so as to
have consistent behav...
0032a284ca8542d169d8e3817142158822c35071 authored 11 months ago
69d47f4029d2a9b10201f0b8ca47aec0e8ec9565 authored 11 months ago
This adds publisher root class registration and some docstrings.
beba98628aaa9637965b6b3a66df5e36d1cb26f4 authored 11 months ago7adf457f78259f596f572bf967e8373fa7d89bbe authored 11 months ago
By trying an ad hoc method of each registered root publisher class.
a4d74e05db2961ff549b430e7e24545479acf516 authored 11 months agoSo that the collection of such classes can be queried.
0cacfc8956f68ff4a82fbe97e3426aba4a33df43 authored 11 months agoThe docstring renders them unnecessary.
5f4878f6a0af83042b3763e916c3d84d81f835be authored 11 months agoUse publisher root classes
ff354352701481f144b2f060b655ce64fe5b8723 authored 11 months agof3e77ff7a5f972fb42fcb89e59c6ac7fb5bff735 authored 11 months ago
853e1dae3210c6c6d4b8963e30fe04b82f809cc7 authored 11 months ago
9c9d29807d4964323658d247cb13c861f8d1aa32 authored 11 months ago
With no checks other than those in the original code (e.g. specifying a plain
file will fail str...
Consistent with the existing helper that only checks for filesystem paths.
1ff5156d369f7eb665d6400d929ee049ca35e8b5 authored 11 months ago50e120ad11b1974efaf1a712f54c013e17eebf47 authored 11 months ago
The relative path is converted to the root's representation (also for
sane/canonical representat...
Define it for the dir publisher root to pull watching files out of main
publisher code.
For non-directory publisher roots, it may make no sense to have a path that
crosses from the rea...
However, uses of that may be an indication that more changes are needed there
to decouple code f...
a16b36915e43761c08628ff32712e59a41832c4e authored 12 months ago
Including internal check for valid dataset relative path.
0c5ed850167d318fd1d70899e9b3e874a72e3563 authored 12 months agoc217116a62959a4c007387eb7b31e3ad70e7bdda authored 12 months ago
And start using that at the publisher.
d23b991ff963000d4c3184dd43db84c861300098 authored 12 months ago21d89d032026ad522042ed9288bbe73c087b13a0 authored 12 months ago
With some potential abstract methods to define its interface.
169b2804f7cd701c8503dec7037d6f11fcb86abe authored 12 months agoCarry compression params on import/export.
This updates `cat2export` and (especially) `cat2im...
9b762544af6431e8c5493a5adfa38bbbf1b46bcb authored 12 months ago81a76060be34ca8d0450c431316bb24017cee350 authored 12 months ago
600613b375001ac8e558bca100422b69d36205f0 authored 12 months ago
Using the same technique as when exporting arrays.
f81ae815efb852de391765426eb4be692c4ada31 authored 12 months ago2c23c53ef82629669454cefc38393cee0aac1978 authored 12 months ago
The technique to keep memory usage low and not create an array each time is
similar to import co...
They are `Iterator` (they can be iterated directly), not `Iterable`.
2fff6ecdc6a00e540dfeb304bfcccaadf456f3d2 authored 12 months agoa9a23d8734535b4afcf9dc66046e8b88586d534a authored 12 months ago
After creating an empty dataset.
730dedf64d25821379cbef3d6dd36215624c1fc6 authored 12 months agoNo need for it to be that nested.
Also for similarity with export code.
5984cf035abe7cbfb35ddfbfff14e560c133b570 authored 12 months ago
They are redundant now, but they will be needed when an empty dataset is
created, so include the...
c5c7a9bf3c7766e83db1f48ad6eadba12435ac2b authored 12 months ago
For similarity with import code. Getting the creator does not perform any
changes, but it still...
For similarity with import code.
b8e45ec3c0e3a0c0d5fdb7ff35dc8739a6e81267 authored 12 months agoFor similarity with import code.
2764ac0d373dbc62efb76fab956c082e723113ef authored 12 months agoe504fa43c8879bc4b7d768dc46757ad49d5a0d65 authored 12 months ago
1bf694e671263c6a8b858db70d4500c360590579 authored 12 months ago
f2e1f3b4c9614d45f7b8215e2966cbf925d314e8 authored 12 months ago
c9fd31b3ba3021f8a38fca0668b462905785914d authored 12 months ago
04e1c9929b1fdaf998c09c42182ab421ec861ff1 authored 12 months ago
5e504e55141ad40e5356930ee6ee49b439b42984 authored 12 months ago
6a6d77eda228235e0aff8853e22ce537aed3a0e7 authored 12 months ago
Since they are only used for that. Also, remove related, obsolete todo
comment.
df4b3f658dd038e2493d8bd3f806313b7f3c8fcb authored 12 months ago
9c493bd88986fd702c4ad246aef70daa5b3e6c40 authored 12 months ago
For consistency.
35153a93450593afa8c85968e4a14510530dcbd3 authored 12 months agof32f544442c7282c82d158ef68b141f88daa0c5d authored 12 months ago
00696bf91e32e904b7d59507a58e506ef64edbc2 authored 12 months ago
Especially for the case of a 1D array where the blockshape was set, since no
chunkshape is store...
c3b13f4272de47a7e3506b3c762b268e97399ea4 authored 12 months ago
243c7c0d9ab2000dcd462ef2e046d0adce7473b8 authored 12 months ago
For Blosc2 arrays, frames and compressed files (plain files still use the
default).
If other filters (like HDF5 shuffle) are enabled, the schunk may be
scrambled (and there is no w...
f20b68a70b9714a1f541441d7ad57ffa33731626 authored 12 months ago
e2a6c4c058ea98afb119d5e4d6d479b47eef6cae authored 12 months ago
b81d054772971fb028aa7afc3f010cc9907cc872 authored 12 months ago
Since nothing has been created yet on disc.
841c44a851376addd037d5bb85a0a4a68c48dee3 authored 12 months agoebaeccf222b39b4d83188e888905cee282bb20ba authored 12 months ago
3d520b478585acb4d116c1a9d70887cb4769c843 authored 12 months ago
fa3e27455a5bf19f23c41cce8ad87ec1fffdd332 authored 12 months ago
c85d5c3e61237b69485aafef3289872cc1f92908 authored 12 months ago
2a581e8f343cdc59dee670d8504fa8c11d3837d1 authored 12 months ago
Otherwise, getting `(array, schunk)` was still prone to segmentation faults if
the GC decided to...
This hides Blosc2 arguments from the caller, and storage issues from Blosc2
conversion machinery...
a96a1c971f7be91ab2adaeacec6355ef6044b11a authored 12 months ago
9fbcf776d8fea702134ca503111a6b7dffb5dce6 authored 12 months ago
9c89764e3489af86158786c436ac9ab319f8bfdb authored 12 months ago
ea002f088f17277ff6ef03b083c852094a4fd87f authored 12 months ago
The empty array already had chunks which were pushed back by the insertion
operation, so they we...
c8b07c2a6510978bd3c82f41549e2bc9f22270df authored 12 months ago
3b0ca92fab7ce8ee4233e3a37b10002fa2bd8b60 authored 12 months ago
Source data is still slurped and re-compressed into memory to create the
chunks to be stored.
738266cb2ae2cddb7aad6d7f59b5cd57d2c775ac authored 12 months ago
7b6bc0fcfcbb62ed41fbf6285efb618f18eec9d8 authored 12 months ago
e74fd489349eef9a91ba57ee6fca90081b7b7732 authored 12 months ago
Add `cat2export tool` (Caterva to HDF5).
This adds the `caterva2.tools.cat_to_hdf5` a.k.a. `c...
f1af7e49fbf17db2731dac153984bd4dbdc450d1 authored 12 months ago6c5dc89a6816d5bd22b5d42fb4d2e5e801f8406f authored 12 months ago
Missing example subsection.
6ea647cbbaeb457900bcd6ec83de4cbf9e5ad8c4 authored 12 months ago3a25a3dda7b4a01f8aa13a019a7333d33d3b76c4 authored 12 months ago
936519d22a0ba487e18159238106634c4f69b67d authored 12 months ago
Undo root being part of dataset path in clients
9960caca52849eee111cae723deb11095238be79 authored 12 months ago
So that exported dataset enjoy some compression instead of being stored
completely decompressed....
785200d4bccb65b8c01016a579ad75236f76199e authored 12 months ago
6f251fadd11c18302f1b91746ecf0efce9377bd4 authored 12 months ago
acc31d70c0793b82c1bfdab93496022c4f341743 authored 12 months ago
aac88f4a0364bcf467cc9d2df1b5c4adaf57e423 authored 12 months ago
f23bcf5d7a30ccd6ff4a4b2a87657618224d50b4 authored 12 months ago
176f2bcceeb7979bd5f96c76ee709dc019978963 authored 12 months ago
Since it depends on the kind of dataset exported.
47adf5f2f11378f2b1eee380ca2c9ce1e58dc182 authored 12 months ago2b2ce1f7904ab372ca813c593e34b98d9f18947e authored 12 months ago