Module¶

Package for using multiple resources while observing multiple RateLimits.

class multi_rate_limit.DayRateLimit(resource_limit: int, period_in_days=1.0)¶: Variant of RateLimit. Specify duration in days.

class multi_rate_limit.FilePastResourceQueue(len_resource: int, longest_period_in_seconds: float)¶

Class to manage resource usage with memory and file(Optional).

_time_resource_queue¶

Queue to manage resource usage.

Type:: deque[Tuple[float, List[int]]]

_longest_period_in_seconds¶

Information before this is forgotten.

Type:: float

_file_path¶

File name to use when you want to reuse resource usage information in another execution. If the file does not exist, it will be created automatically.

Type:: Optional[str]

async add(use_time: float, use_resources: List[int]) → None¶

Add resource usage information.

Cancel from MultiRateLimit is protected by shield, so it can be executed until the end unless you cancel it yourself. On the other hand, there is a possibility that another function will be called before completion, so if you use await internally, you need to properly make newcoming functions wait so that the integrity is not compromised.

In this implementation, if you try to add information with a time before the existing last resource usage information, the resource will be forced to be used at the time of the last resource usage.

Parameters:

use_time (float) – Resource usage time compatible with time.time().
use_resources (List[int]) – Resource usage amounts. The length of list must be same as the number of resources. Each resource usage amaount must not be negative.

async classmethod create(len_resource: int, longest_period_in_seconds: float, file_path: str | None = None)¶

Create a queue to manage past resouce usages with memory and file(Optional).

Parameters:

len_resource (int) – Number of resource types.
longest_period_in_seconds (float) – How far in the past should information be remembered?
file_path (Optional[str], optional) – File name to use when you want to reuse resource usage information in another execution. If the file does not exist, it will be created automatically. Defaults to None.

Returns:

An class to manage resource usage with memory and file(Optional).

Return type:

_type_

pos_accum_resouce_within(order: int, amount: int) → int¶

Returns the last index in the queue when resource usage falls within the specified amount.

Returns the latest index at which the cumulative amount of resource usage exceeds the specified amount, going back from the end of the queue.

Parameters:

order (int) – The order of resource.
amount (int) – The specified amount.

Returns:

The last index in the queue when resource usage falls within the specified amount.

Return type:

int

pos_time_after(time: float) → int¶

Returns the position on the first resource information queue after the specified time.

If there is no match, return the queue length.

Parameters:: time (float) – The specified time.
Returns:: The position on the first resource information queue after the specified time.
Return type:: int

async sum_resource_after(time: float, order: int) → int¶

Returns the amount of resources of specified order used after the specified time.

If the specified time is before the last resource use beyond the period passed at the constructor, it is okay to return incorrect information. This allows old information unrelated to resource limit management to be forgotten.

Parameters:

time (float) – The specified time compatible with time.time().
order (int) – The order of resource.

Returns:

The amount of resources of specified order used after the specified time.

Return type:

int

async term() → None¶: Called when finished. Do nothing.

async time_accum_resource_within(order: int, amount: int) → float¶

Returns the last timing when resource usage falls within the specified amount.

Returns the latest timing at which the cumulative amount of resource usage exceeds the specified amount, going back from the current time.

Parameters:

order (int) – The order of resource.
amount (int) – The specified amount.

Returns:

The last timing compatible with time.time() when resource usage falls within the specified amount.

Return type:

float

class multi_rate_limit.HourRateLimit(resource_limit: int, period_in_hours=1.0)¶: Variant of RateLimit. Specify duration in hours.

class multi_rate_limit.IPastResourceQueue¶

Interface to customize how used resources are managed.

abstract async add(use_time: float, use_resources: List[int]) → None¶

Add resource usage information.

Cancel from MultiRateLimit is protected by shield, so it can be executed until the end unless you cancel it yourself. On the other hand, there is a possibility that another function will be called before completion, so if you use await internally, you need to properly make newcoming functions wait so that the integrity is not compromised.

Parameters:

use_time (float) – Resource usage time compatible with time.time().
use_resources (List[int]) – Resource usage amounts. The length of list must be same as the number of resources. Each resource usage amaount must not be negative.

abstract async sum_resource_after(time: float, order: int) → int¶

Returns the amount of resources of specified order used after the specified time.

If the specified time is before the last resource use beyond the period passed at the constructor, it is okay to return incorrect information. This allows old information unrelated to resource limit management to be forgotten.

Parameters:

time (float) – The specified time compatible with time.time().
order (int) – The order of resource.

Returns:

The amount of resources of specified order used after the specified time.

Return type:

int

abstract async term() → None¶

Called when finished.

Can be used to persist unrecorded data. It is not guaranteed that it will be called, so you should make sure that it does not cause a fatal situation even if it is not called.

abstract async time_accum_resource_within(order: int, amount: int) → float¶

Returns the last timing when resource usage falls within the specified amount.

Returns the latest timing at which the cumulative amount of resource usage exceeds the specified amount, going back from the current time.

Parameters:

order (int) – The order of resource.
amount (int) – The specified amount.

Returns:

The last timing compatible with time.time() when resource usage falls within the specified amount.

Return type:

float

class multi_rate_limit.MinuteRateLimit(resource_limit: int, period_in_minutes=1.0)¶: Variant of RateLimit. Specify duration in minutes.

class multi_rate_limit.MultiRateLimit¶

Class for using multiple resources while observing multiple RateLimits.

_limits¶

Resource limits.

Type:: List[List[RateLimit]]

_past_queue¶

Executed resource usage manager.

Type:: IPastResourceQueue

_current_buffer¶

Running resource usage manager.

Type:: CurrentResourceBuffer

_next_queue¶

Waiting resource usage manager.

Type:: NextResourceQueue

_loop¶

Cached event loop.

Type:: AbstractEventLoop

_in_process¶

Asynchronous execution tasks for internal processing.

Type:: Optional[Task]

_terminated¶

Whether term() has been called.

Type:: bool

cancel(number: int, auto_close: bool = False) → Tuple[List[int], Coroutine[Any, Any, Tuple[Tuple[float, List[int]] | None, Any]]] | None¶

Cancel the reservation of a waiting coroutine.

This process automatically cancels the future of the ticket.

Parameters:

number (int) – Ticket number.
auto_close (bool, optional) – If true, automatically close the canceled coroutine. Defaults to False. This coroutine can be reused. But you don’t reuse it, Runtime warning will occure when the program finish. Automatic close can suppress this warning.

Raises:

Exception – If already terminated.

Returns:

Reserved resource amount and coroutine object.

Return type:

Optional[Tuple[List[int], Coroutine[Any, Any, Tuple[Optional[Tuple[float, List[int]]], Any]]]]

async classmethod create(limits: List[List[RateLimit]], past_queue_factory: Callable[[int, float], Coroutine[Any, Any, IPastResourceQueue]] = None, max_async_run=1)¶

Create an object for using multiple resources while observing multiple RateLimits.

Parameters:

limits (List[List[RateLimit]]) – Resource limits.
past_queue_factory (Callable[[int, float], Coroutine[Any, Any, IPastResourceQueue]], optional) – Pass the factory method to make the executed resource usage manager. The default is None, in which case it is managed only in memory.
max_async_run (int, optional) – Maximum asynchronous concurrency. Defaults to 1.

Raises:

ValueError – If the resource limit array length is 0, or if any value of the resource limit or max_async_run is non-positive.

Returns:

Object for using multiple resources while observing multiple RateLimits.

Return type:

_type_

reserve(use_resources: List[int], coro: Coroutine[Any, Any, Tuple[Tuple[float, List[int]] | None, Any]]) → ReservationTicket¶

Schedules the task and returns a ticket to receive the result.

Unless explicitly stated in the return value or exception parameter of coroutine, the use_resources of this function are considered to have been consumed at the end of coroutine execution. If you want to change this behavior because you cannot know the exact resource consumption until after execution, please override the resource consumption timing and amount using coroutine’s return value or ResourceOverwriteError parameter.

The return value of coroutine is in the following format. ((use_time, [use_resource1, use_resource2,,,]), return_value_to_user) If you do not want to overwrite, please use the followin format. (None, return_value_to_user)

Parameters:

use_resources (List[int]) – Resource reservation amount.
coro (Coroutine[Any, Any, Tuple[Optional[Tuple[float, List[int]]], Any]]) – Coroutine object that is the process to reserve

Raises:

Exception – If already terminated.
ValueError – In case of resources list length mismatch or any single resource reservation exceeds its limit.
ValueError – If the passed process is not a coroutine.

Returns:

Ticket to receive the result.

Return type:

ReservationTicket

runnings() → int¶

Returns the number of currently running coroutines.

Returns:: The number of currently running coroutines.
Return type:: int

async stats(current_time: float | None = None) → RateLimitStats¶

Returns resource usage.

Parameters:: current_time (Optional[float], optional) – The current time. The default is None, in which case the result of time.time() is used.
Raises:: Exception – If already terminated.
Returns:: Resource usage.
Return type:: RateLimitStats

async term(auto_close: bool = False) → List[Coroutine[Any, Any, Tuple[Tuple[float, List[int]] | None, Any]]]¶

End processing.

Cancels all waiting processes and waits for all currently running processes to finish and for resource managers that have already been executed to terminate.

Parameters:: auto_close (bool, optional) – If true, automatically close the canceled coroutine. Defaults to False. This coroutine can be reused. But you don’t reuse it, Runtime warning will occure when the program finish. Automatic close can suppress this warning.
Raises:: Exception – If already terminated.
Returns:: Waiting coroutines.
Return type:: List[Coroutine[Any, Any, Tuple[Optional[Tuple[float, List[int]]], Any]]]

termed() → bool¶

Returns whether this object is termed.

Returns:: Whether this object is termed.
Return type:: bool

waiting_numbers() → KeysView[int]¶

Returns waiting coroutines’ reservation numbers.

Returns:: Waiting coroutines’ reservation numbers.
Return type:: KeysView[int]

waitings() → int¶

Returns the number of waiting coroutines.

Returns:: The number of waiting coroutines.
Return type:: int

class multi_rate_limit.RateLimit(resource_limit: int, period_in_seconds: float)¶

Class to define a single resource limit.

_resource_limit¶

Resource limit that can be used within the period.

Type:: int

_period_in_seconds¶

Resource limit period.

Type:: float

property period_in_seconds: float¶

Return the resource limit period in seconds.

Returns:: Resource limit period in seconds.
Return type:: float

property resource_limit: int¶

Return the resource limit that can be used within the period.

Returns:: Resource limit that can be used within the period.
Return type:: int

class multi_rate_limit.RateLimitStats(limits: List[List[RateLimit]], past_uses: List[List[int]], current_uses: List[int], next_uses: List[int])¶

Class that represents resource usage status.

limits¶

Resource limits

Type:: List[List[RateLimit]]

past_uses¶

Total resource usage that has been executed for each resource limit. (For 1 minute limit, resource usage for the past 1 minute.)

Type:: List[List[int]]

current_uses¶

Total running resource usage for each resource.

Type:: List[int]

next_uses¶

Total waiting resource usage for each resource.

Type:: List[int]

current_use_percents() → List[List[float]]¶

Returns the percentage of total executed and running resource usage relative to each resource limit.

Returns:: The percentage of total executed and running resource usage relative to each resource limit.
Return type:: List[List[float]]

next_use_percents() → List[List[float]]¶

Returns the total usage of executed, running, and waiting resources as a percentage of each resource limit.

Returns:: The total usage of executed, running, and waiting resources as a percentage of each resource limit.
Return type:: List[List[float]]

past_use_percents() → List[List[float]]¶

Returns the percentage of total executed resource usage against each resource limit.

Returns:: The percentage of total executed resource usage against each resource limit.
Return type:: List[List[float]]

class multi_rate_limit.ReservationTicket(reserve_number: int, future: Future[Any])¶

Class for receiving the results of processing executed through MultiRateLimit.

reserve_number¶

Number to interrupt execution.

Type:: int

future¶

Future to receive execution results.

Type:: Future[Any]

exception multi_rate_limit.ResourceOverwriteError(use_time: float, use_resources: List[int], cause: Exception)¶

Error to customize resource usage.

You can use this error when you want to change the amount or timing of resource usage while returning an exception from within a coroutine that applies RateLimit.

use_time¶

Resource usage time compatible with time.time() to be overwritten.

Type:: float

use_resources¶

Resource usage amounts to be overwritten. The length of list must be same as the number of resources. Each resource usage amaount must not be negative.

Type:: List[int]

cause¶

Wrap and pass the exception you originally wanted to return.

Type:: Exception

class multi_rate_limit.SecondRateLimit(resource_limit: int, period_in_seconds=1.0)¶: Alias of RateLimit. Specify duration in seconds.

Module¶

multi-rate-limit

Navigation

Related Topics