Skip to content

Add configurable retries for getInfo calls and document retry tuning#304

Open
jdbcode wants to merge 1 commit intomainfrom
feat/retry-getinfo-calls
Open

Add configurable retries for getInfo calls and document retry tuning#304
jdbcode wants to merge 1 commit intomainfrom
feat/retry-getinfo-calls

Conversation

@jdbcode
Copy link
Copy Markdown
Member

@jdbcode jdbcode commented Apr 15, 2026

Summary

Adds configurable retry and backoff handling for Earth Engine getInfo() calls (metadata operations) to address issue #299, where users experience 429 rate-limit errors under quota pressure.

Complements existing getitem_kwargs (pixel fetch retries) with new getinfo_kwargs (metadata call retries), giving users fine-grained control over retry behavior for both data and control-plane operations.

🚀 Default Behavior

  • getinfo_kwargs: {'max_retries': 6, 'initial_delay': 1000} — Conservative 1000ms initial delay to reduce setup-time retry bursts against EE metadata endpoints
  • Affected operations: get_info property, extract_grid_params(), geometry_to_bounds()
  • Pattern: Same exponential backoff + jitter as pixel retries, but user-overridable at all entry points

API Changes

xee.ext.open_dataset()

# New parameter
getinfo_kwargs: Dict[str, int] | None = None

xee.helpers.extract_grid_params()

# New parameter
getinfo_kwargs: Dict[str, int] | None = None

xee.helpers.geometry_to_bounds()

# Now accepts optional getinfo_kwargs
geometry_to_bounds(geometry, scale=None, getinfo_kwargs=None)

Test Coverage

  • Unit tests (4 new): Configuration defaults, override merging, and retry wiring validation
  • Integration tests (2 new, 35/35 passing): Real EE API validation of config propagation
  • All tests passing: 30 unit + 35 integration in ~408 seconds

Documentation Updates

  • FAQ: Added detailed 429 troubleshooting section with configuration example
  • Performance Guide: Added "Retry Tuning" subsection with defaults rationale and tuning guidance
  • User Guide: Added "Configure Retries" section with practical examples

Migration Notice ⚠️

This work is being added post-v0.1.0 (merged to main a week ago). If you're testing this branch, your workflow may need to be updated to use the new API. Please see the migration guide for details on API changes introduced in v0.1.0.

Validation Request

@adkinsjd — We'd appreciate your feedback on whether this resolves the 429 errors in your workflow. If you test this branch, please report:

  1. Outcome: Are failures eliminated or reduced?
  2. Tuning: Did you need to adjust the retry settings? If so, what values worked?
  3. Endpoint: Which Earth Engine endpoint was throwing 429s? (metadata/pixel data/both)

Install: pip install --no-cache-dir "git+https://github.com/google/Xee.git@feat/retry-getinfo-calls"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants