pasteur.utils.logging.MlflowHandler

class pasteur.utils.logging.MlflowHandler(name='user', logdir='_logs', interval=5)

A handler class based on StreamHandler and MemoryHandler, which buffers the records using a StringIO and dumps that StringIO every N records into an mlflow artifact.

@Warning: currently doesn’t have a size limit

Attributes

name
terminator

Methods

acquire()	Acquire the I/O thread lock.
addFilter(filter)	Add the specified filter to this handler.
close()	Tidy up any resources used by the handler.
close_all()	Call this function to close the mlflow handlers before ending a run if you don't plan to reuse them.
createLock()	Acquire a thread lock for serializing access to the underlying I/O.
emit(record)	Do whatever it takes to actually log the specified logging record.
filter(record)	Determine if a record is loggable by consulting all the filters.
flush()	Ensure all logging output has been flushed.
flush_all()
format(record)	Format the specified record.
get_name()
handle(record)	Conditionally emit the specified logging record.
handleError(record)	Handle errors which occur during an emit() call.
release()	Release the I/O thread lock.
removeFilter(filter)	Remove the specified filter from this handler.
reset()	Flushes and removes the current buffer of the handler, allowing it to be reused for a new run.
reset_all()	Flushes and cleans up all handlers so that they can be used for a new run.
setFormatter(fmt)	Set the formatter for this handler.
setLevel(level)	Set the logging level of this handler.
set_name(name)

acquire()

Acquire the I/O thread lock.

addFilter(filter)

Add the specified filter to this handler.

close()

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

Return type:

None

static close_all()

Call this function to close the mlflow handlers before ending a run if you don’t plan to reuse them.

createLock()

Acquire a thread lock for serializing access to the underlying I/O.

emit(record)

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

filter(record)

Determine if a record is loggable by consulting all the filters.

The default is to allow the record to be logged; any filter can veto this and the record is then dropped. Returns a zero value if a record is to be dropped, else non-zero.

Changed in version 3.2: Allow filters to be just callables.

flush()

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

static flush_all()

format(record)

Format the specified record.

If a formatter is set, use it. Otherwise, use the default formatter for the module.

get_name()

handle(record)

Conditionally emit the specified logging record.

Emission depends on filters which may have been added to the handler. Wrap the actual emission of the record with acquisition/release of the I/O thread lock. Returns whether the filter passed the record for emission.

handleError(record)

Handle errors which occur during an emit() call.

This method should be called from handlers when an exception is encountered during an emit() call. If raiseExceptions is false, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The record which was being processed is passed in to this method.

property name

release()

Release the I/O thread lock.

removeFilter(filter)

Remove the specified filter from this handler.

reset()

Flushes and removes the current buffer of the handler, allowing it to be reused for a new run.

static reset_all()

Flushes and cleans up all handlers so that they can be used for a new run.

setFormatter(fmt)

Set the formatter for this handler.

setLevel(level)

Set the logging level of this handler. level must be an int or a str.

set_name(name)

terminator = '\n'