-
-
Notifications
You must be signed in to change notification settings - Fork 215
[ENH] V1 → V2 API Migration - datasets #1608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
JATAYU000
wants to merge
142
commits into
openml:main
Choose a base branch
from
JATAYU000:dataset_resource
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
142 commits
Select commit
Hold shift + click to select a range
0159f47
set up folder structure and base code
geetu040 58e9175
Merge branch 'main' into migration
fkiraly bdd65ff
Merge branch 'main' into migration
geetu040 52ef379
fix pre-commit
geetu040 f7ba710
Merge base migration pr, ruff
JATAYU000 5dfcbce
refactor
geetu040 2acbe99
implement cache_dir
geetu040 af99880
refactor
geetu040 74ab366
Merge branch 'main' into pr/1576
fkiraly 8964517
edit, fork, delete updated
JATAYU000 1c2fa99
Added features, updated list
JATAYU000 18e85de
Merge commit pull/1576 into dataset_resource
JATAYU000 9bcbcb3
Refactor functions, except get
JATAYU000 96df5e3
Remove circular import using lazy import
JATAYU000 c955f43
Modify reviews, feature and qualities
JATAYU000 4c75e16
undo changes in tasks/functions.py
geetu040 5762185
Merge branch 'main' into migration
geetu040 3ad7268
Merge base pr
JATAYU000 3e7c415
Download methods
JATAYU000 8933cd8
Refactored functions
JATAYU000 dfa0ab7
Update todos, topic endpoints
JATAYU000 7e9bc1f
Merge branch 'main' into migration
geetu040 c603383
add tests directory
geetu040 ff6a8b0
use enum for delay method
geetu040 f01898f
implement cache
geetu040 5c4511e
refactor clients
geetu040 05dec9c
fixed minio methods and edit migration
JATAYU000 2012a72
Merge base pr
JATAYU000 35f450a
Updated Minio, download methods
JATAYU000 5080752
checksum for arff download
JATAYU000 24582ef
use cache, new enpoints
JATAYU000 d789e83
lazy imports
JATAYU000 43276d2
fix import in resources/base.py
geetu040 1206f69
refactor and add exception handling
geetu040 4948e99
refactor resources/base/
geetu040 a354167
implement delete
geetu040 1fe7e3e
implement publish and minor refactoring
geetu040 8d51fea
Fixes some tests
JATAYU000 bd85ec4
fix conflicts
JATAYU000 54a3151
implement tag/untag
geetu040 2b6fe65
implement fallback
geetu040 f4d6e75
code 103, dataset download
JATAYU000 0c16547
New tests partial
JATAYU000 fa53f8d
add test_http.py
geetu040 2b2db96
add uses_test_server marker
geetu040 f4c739d
Update delete test
JATAYU000 b87feb5
Merge commit 'refs/pull/1576/head' into dataset_resource
JATAYU000 c9617f9
implement reset_cache
geetu040 6bb20c3
Post update, v2 tests additions
JATAYU000 7d7fa43
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 5bc37b8
fixes with publish/delete
geetu040 08d9916
fix cache_key in tests
geetu040 8caba11
update _not_supported
geetu040 b4a817a
Merge commit '/pull/1576'
JATAYU000 1913c10
add 'get_api_config' skeleton method
SimonBlanke 7681949
remove 'APISettings'
SimonBlanke 01840a5
impl. 'get_api_config'
SimonBlanke 26ed4c1
add singleton pattern for settings
SimonBlanke c588d0c
add 'reset_settings'
SimonBlanke b6ff720
remove unused code
SimonBlanke 80d5afc
reimplement usage of v1 settings config
SimonBlanke f47112c
first try v2, fallback to v1 if not available
SimonBlanke d44cf3e
reimplement singelton without the use of 'global'
SimonBlanke ea7dda1
add explanations
SimonBlanke f0e5947
change usage of settings to new impl.
SimonBlanke edcd006
add explanations
SimonBlanke cde0aae
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] aa1e560
move to config: APIVersion, ResourceType
geetu040 06b8497
remove api_context entirely
geetu040 384da91
major refactor
geetu040 1878138
more refactoring with setup/
geetu040 dc26e01
implement APIBackend as controller
geetu040 e2d059b
move enums
geetu040 d156ad4
module level imports
geetu040 d7a3788
module level import for _backend
geetu040 b5b9ef6
module level import for tests
geetu040 68820fe
Merge branch 'main' into migration
geetu040 7288284
New tests,cache exception in download
JATAYU000 567eca4
add test: test_tag_and_untag
geetu040 d23790b
Merge, post fix
JATAYU000 95e8890
md5_checksum, and request fix
JATAYU000 16c9251
Merge branch 'openml:main' into dataset_resource
JATAYU000 23fe19b
Merge commit /pull/1576
JATAYU000 be29dc9
Merge branch 'dataset_resource'
JATAYU000 b2287c3
implement get/set_config_values
geetu040 b7e285e
improve APIBackend.set_config_values
geetu040 fd43c48
use LegacyConfig
geetu040 f4aab6b
Revert "use LegacyConfig"
geetu040 d43cf86
implement _sync_api_config
geetu040 3e323ed
update tests with _sync_api_config
geetu040 9195fa6
rename config: timeout -> timeout_seconds
geetu040 5342eec
use timedelta for default ttl value
geetu040 adc0e74
update tests, adds v2/fallback
geetu040 bfb2d3e
add MinIOClient in TestBase
geetu040 707e1f1
publish,tag methods need testing
JATAYU000 cabaecf
fix linting for builder
geetu040 79cf49c
new migration tests
JATAYU000 5c8791a
Merge /1576
JATAYU000 85c1113
fix unbound variables: "code", "message"
geetu040 39bf86a
use requests.Session()
geetu040 7b66677
remove "timeout_seconds" entirely
geetu040 d2224c4
update/refactor tests
geetu040 9608c36
remove unused current_api_version from TestAPIBase
geetu040 f6bc7f7
make TestAPIBase inherit TestBase
geetu040 baa3a38
nits: test classes
geetu040 29c93d1
Review changes, new tests
JATAYU000 7674b3a
Merge bse migration
JATAYU000 ddb0774
Doc strings
JATAYU000 52b93fe
minor fix in _sync_api_config
geetu040 ec9477f
chore: rerun CI
geetu040 cea6188
delete mock, decorator
JATAYU000 3d4e84d
Merge base
JATAYU000 839bd33
delete url in test
JATAYU000 8417349
New test design
JATAYU000 10d134a
remove duplicates in _api/resources/__init__.py
geetu040 935f0f4
implement HTTPClient.download and add tests
geetu040 9514df8
add docstrings
geetu040 09f9ad6
Review changes
JATAYU000 0b52427
Merge base pr
JATAYU000 53bee94
update minio
geetu040 33b4ca0
make delay functions static
geetu040 a6b9a45
rename: retry_raise_e -> exception
geetu040 f924b32
use context-manager for requests.Session
geetu040 541b0f2
remove "assert response is not None"
geetu040 acb173f
verify checksum before caching
geetu040 3e8d1f0
update tests
geetu040 f83bdb5
minor fix in ResourceV1API.untag
geetu040 969c7d8
Merge branch 'main' into dataset_resource
JATAYU000 2a42712
remove cache.ttl
geetu040 001caad
replace config.cache.dir with config.cache_dir
geetu040 fb38a2d
make HTTPClient.cache compulsory
geetu040 03c4ca9
remove unused OpenMLCacheRequiredError
geetu040 8d708fd
implement and use TestAPIBase._create_resource
geetu040 4f75bba
make ResourceAPI.minio compulsory
geetu040 164f66f
Merge branch 'main' into migration
geetu040 c4dae43
rename: use_cache -> enable_cache; reset_cache -> refresh_cache
geetu040 36c20a2
use server config from TestBase
geetu040 ab3c1eb
tests: mock HTTP post calls to prevent race conditions
geetu040 0fc3c74
Merge bse_migration into dataset_resource
JATAYU000 741a66b
rename cache params
JATAYU000 81dff8d
Merge branch 'dataset_resource'
JATAYU000 27ac86f
Minio assertions, other reviews
JATAYU000 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| from .clients import ( | ||
| HTTPCache, | ||
| HTTPClient, | ||
| MinIOClient, | ||
| ) | ||
| from .resources import ( | ||
| API_REGISTRY, | ||
| DatasetAPI, | ||
| DatasetV1API, | ||
| DatasetV2API, | ||
| EstimationProcedureAPI, | ||
| EstimationProcedureV1API, | ||
| EstimationProcedureV2API, | ||
| EvaluationAPI, | ||
| EvaluationMeasureAPI, | ||
| EvaluationMeasureV1API, | ||
| EvaluationMeasureV2API, | ||
| EvaluationV1API, | ||
| EvaluationV2API, | ||
| FallbackProxy, | ||
| FlowAPI, | ||
| FlowV1API, | ||
| FlowV2API, | ||
| ResourceAPI, | ||
| ResourceV1API, | ||
| ResourceV2API, | ||
| RunAPI, | ||
| RunV1API, | ||
| RunV2API, | ||
| SetupAPI, | ||
| SetupV1API, | ||
| SetupV2API, | ||
| StudyAPI, | ||
| StudyV1API, | ||
| StudyV2API, | ||
| TaskAPI, | ||
| TaskV1API, | ||
| TaskV2API, | ||
| ) | ||
| from .setup import ( | ||
| APIBackend, | ||
| APIBackendBuilder, | ||
| APIConfig, | ||
| Config, | ||
| ConnectionConfig, | ||
| _backend, | ||
| ) | ||
|
|
||
| __all__ = [ | ||
| "API_REGISTRY", | ||
| "APIBackend", | ||
| "APIBackendBuilder", | ||
| "APIConfig", | ||
| "Config", | ||
| "ConnectionConfig", | ||
| "DatasetAPI", | ||
| "DatasetV1API", | ||
| "DatasetV2API", | ||
| "EstimationProcedureAPI", | ||
| "EstimationProcedureV1API", | ||
| "EstimationProcedureV2API", | ||
| "EvaluationAPI", | ||
| "EvaluationMeasureAPI", | ||
| "EvaluationMeasureV1API", | ||
| "EvaluationMeasureV2API", | ||
| "EvaluationV1API", | ||
| "EvaluationV2API", | ||
| "FallbackProxy", | ||
| "FlowAPI", | ||
| "FlowV1API", | ||
| "FlowV2API", | ||
| "HTTPCache", | ||
| "HTTPClient", | ||
| "MinIOClient", | ||
| "ResourceAPI", | ||
| "ResourceV1API", | ||
| "ResourceV2API", | ||
| "RunAPI", | ||
| "RunV1API", | ||
| "RunV2API", | ||
| "SetupAPI", | ||
| "SetupV1API", | ||
| "SetupV2API", | ||
| "StudyAPI", | ||
| "StudyV1API", | ||
| "StudyV2API", | ||
| "TaskAPI", | ||
| "TaskV1API", | ||
| "TaskV2API", | ||
| "_backend", | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| from .http import HTTPCache, HTTPClient | ||
| from .minio import MinIOClient | ||
|
|
||
| __all__ = [ | ||
| "HTTPCache", | ||
| "HTTPClient", | ||
| "MinIOClient", | ||
| ] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.