rest

prefect_databricks.rest

This is a module containing generic REST tasks.

Functions

serialize_model

serialize_model(obj: Any) -> Any

Recursively serializes pydantic.BaseModel into JSON; returns original obj if not a BaseModel.

Args:

• obj: Input object to serialize.

Returns:

• Serialized version of object.

strip_kwargs

strip_kwargs(**kwargs: Dict) -> Dict

Recursively drops keyword arguments if value is None, and serializes any pydantic.BaseModel types.

Args:

• **kwargs: Input keyword arguments.

Returns:

• Stripped version of kwargs.

execute_endpoint

execute_endpoint(endpoint: str, databricks_credentials: 'DatabricksCredentials', http_method: HTTPMethod = HTTPMethod.GET, params: Dict[str, Any] = None, json: Dict[str, Any] = None, **kwargs: Dict[str, Any]) -> httpx.Response

Generic function for executing REST endpoints.

Args:

• endpoint: The endpoint route.
• databricks_credentials: Credentials to use for authentication with Databricks.
• http_method: Either GET, POST, PUT, DELETE, or PATCH.
• params: URL query parameters in the request.
• json: JSON serializable object to include in the body of the request.
• **kwargs: Additional keyword arguments to pass.

Returns:

• The httpx.Response from interacting with the endpoint.

Examples:

Lists jobs on the Databricks instance.

from prefect import flow
from prefect_databricks import DatabricksCredentials
from prefect_databricks.rest import execute_endpoint


@flow
async def example_execute_endpoint_flow():
    endpoint = "/2.1/jobs/list"
    databricks_credentials = await DatabricksCredentials.load("my-block")
    params = {
        "limit": 5,
        "offset": None,
        "expand_tasks": True,
    }
    response = await execute_endpoint(
        endpoint,
        databricks_credentials,
        params=params,
    )
    return response.json()

Classes

HTTPMethod

Available HTTP request methods.