Video

Video Streams

class av.video.stream.VideoStream

Bases: Stream

average_rate

The average frame rate of this video stream.

This is calculated when the file is opened by looking at the first few frames and averaging their rate.

Type:

AVRational

base_rate

The base frame rate of this stream.

This is calculated as the lowest framerate at which the timestamps of frames can be represented accurately. See AVStream.r_frame_rate for more.

Type:

AVRational

decode(Packet packet: Packet | None = None)

Decode a Packet and return a list of VideoFrame.

Return type:

list[VideoFrame]

See also

This is a passthrough to CodecContext.decode().

display_aspect_ratio

The guessed display aspect ratio (DAR) of this stream.

This is calculated from VideoStream.guessed_sample_aspect_ratio().

Type:

AVRational

encode(VideoFrame frame: VideoFrame | None = None)

Encode an VideoFrame and return a list of Packet.

Return type:

list[Packet]

See also

This is mostly a passthrough to CodecContext.encode().

guessed_rate

The guessed frame rate of this stream.

This is a wrapper around av_guess_frame_rate, and uses multiple heuristics to decide what is “the” frame rate.

Type:

AVRational

sample_aspect_ratio

The guessed sample aspect ratio (SAR) of this stream.

This is a wrapper around av_guess_sample_aspect_ratio, and uses multiple heuristics to decide what is “the” sample aspect ratio.

Type:

AVRational

set_display_matrix(matrix)

Set the display matrix written to the container as coded side data.

matrix is a sequence of 9 integers in FFmpeg’s AV_PKT_DATA_DISPLAYMATRIX layout, or None to clear. Must be called before the first frame is encoded. See set_display_rotation() for a higher-level helper.

set_display_rotation(degrees, hflip=False, vflip=False)

Set the container display matrix from a rotation and optional flips.

degrees is a counter-clockwise rotation (matching the value read back from VideoFrame.rotation); hflip / vflip mirror after it. Together these express all eight EXIF orientations. Must be called before the first frame is encoded.

Video Codecs

class av.video.codeccontext.VideoCodecContext

Bases: CodecContext

bits_per_coded_sample

The number of bits per sample in the codedwords. It’s mandatory for this to be set for some formats to decode properly.

Wraps AVCodecContext.bits_per_coded_sample.

Type:

int

chroma_sample_location

The position of the chroma samples relative to the luma samples, as FFmpeg’s raw integer value. ChromaLocation names the values.

Wraps AVCodecContext.chroma_sample_location. FFmpeg spells the same field chroma_location on a frame, and so does PyAV: see VideoFrame.chroma_location.

Type:

int

coded_height
Type:

int

coded_width
Type:

int

color_primaries

Describes the RGB/XYZ matrix of the colorspace, as FFmpeg’s raw integer value. ColorPrimaries names the values.

Wraps AVCodecContext.color_primaries.

Type:

int

color_range

Describes the signal range of the colorspace, as FFmpeg’s raw integer value. ColorRange names the values.

Wraps AVCodecContext.color_range.

Type:

int

color_trc

Describes the linearization function (a.k.a. transformation characteristics) of the colorspace, as FFmpeg’s raw integer value. ColorTrc names the values.

Wraps AVCodecContext.color_trc.

Type:

int

colorspace

Describes the YUV/RGB transformation matrix of the colorspace, as FFmpeg’s raw integer value. Colorspace names the values.

Wraps AVCodecContext.colorspace.

Type:

int

display_aspect_ratio
field_order

The video field order as FFmpeg’s raw integer value.

Wraps AVCodecContext.field_order.

format
framerate

The frame rate, in frames per second.

Type:

AVRational

gop_size

Sets the number of frames between keyframes. Used only for encoding.

Type:

int

has_b_frames
Type:

bool

height
max_b_frames

The maximum run of consecutive B frames when encoding a video.

Type:

int

mb_decision

Macroblock decision mode: 0 (simple), 1 (bits) or 2 (rate distortion).

Wraps AVCodecContext.mb_decision.

Type:

int

pix_fmt

The pixel format’s name.

Type:

str | None

qmax

The maximum quantiser value of an encoded stream.

Wraps AVCodecContext.qmax.

Type:

int

qmin

The minimum quantiser value of an encoded stream.

Wraps AVCodecContext.qmin.

Type:

int

rate

Another name for framerate.

reformatter
refs

The number of reference frames.

Wraps AVCodecContext.refs.

Type:

int

reorder_depth

Raw has_b_frames value from FFmpeg (int, not bool).

After flush_buffers(), FFmpeg may reset the internal reorder heuristic. Set this to the known reorder depth after seeking to avoid dropped hierarchical B-frames.

sample_aspect_ratio
sw_format

For a hardware context (e.g. format.name == "cuda"), the underlying software pixel format (nv12, p010le, …). None for a regular software context.

Type:

VideoFormat | None

width

Video Formats

class av.video.format.VideoFormat

Bases: object

>>> format = VideoFormat('rgb24')
>>> format.name
'rgb24'
bits_per_pixel
chroma_height(luma_height=0)

Height of a chroma plane relative to a luma plane.

Parameters:

luma_height (int) – Height of the luma plane; defaults to self.height.

chroma_width(luma_width=0)

Width of a chroma plane relative to a luma plane.

Parameters:

luma_width (int) – Width of the luma plane; defaults to self.width.

components
has_palette

Pixel format has a palette in data[1], values are indexes in this palette.

height
is_bayer

The pixel format contains Bayer data.

is_big_endian

Pixel format is big-endian.

is_bit_stream

All values of a component are bit-wise packed end to end.

is_planar

At least one pixel component is not in the first data plane.

is_rgb

The pixel format contains RGB-like data (as opposed to YUV/grayscale).

name

Canonical name of the pixel format.

padded_bits_per_pixel
width
class av.video.format.VideoFormatComponent

Bases: object

bits

Number of bits in the component.

height

The height of this component’s plane.

Requires the parent VideoFormat to have a height.

index
is_alpha

Is this component an alpha channel?

is_chroma

Is this component a chroma channel?

is_luma

Is this component a luma channel?

plane

The index of the plane which contains this component.

width

The width of this component’s plane.

Requires the parent VideoFormat to have a width.

Video Frames

class av.video.frame.CudaContext

Bases: object

A reusable FFmpeg CUDA context for frames imported with DLPack.

Parameters:

  • device_id (int) – CUDA device ordinal.

  • primary_ctx (bool) – Retain the device’s primary CUDA context.

  • current_ctx (bool) – Wrap the CUDA context current on the calling thread when this object is first used. This is mutually exclusive with primary_ctx.

  • cuda_stream (int) – Optional raw CUstream pointer for FFmpeg CUDA operations, including NVENC input and output. The caller owns the stream and must keep it alive as long as this context can be used. Nonzero stream pointers are currently supported only for logical CUDA device 0. To use another physical GPU, expose it as logical device 0 via CUDA_VISIBLE_DEVICES.

cuda_stream
current_ctx
device_id
primary_ctx
class av.video.frame.VideoFrame

Bases: Frame

A single video frame.

Parameters:

  • width (int) – The width of the frame.

  • height (int) – The height of the frame.

  • format (VideoFormat or str.) – The format of the frame.

>>> frame = VideoFrame(1920, 1080, 'rgb24')

Structural

VideoFrame.width

Width of the image, in pixels.

VideoFrame.height

Height of the image, in pixels.

VideoFrame.format

The VideoFormat of the frame.

VideoFrame.planes

A tuple of VideoPlane objects.

Types

VideoFrame.key_frame

Is this frame a key frame?

Reads the AV_FRAME_FLAG_KEY bit of AVFrame.flags. FFmpeg removed the AVFrame.key_frame field this used to wrap.

VideoFrame.interlaced_frame

Is this frame an interlaced or progressive?

VideoFrame.pict_type

Returns an integer that corresponds to the PictureType enum.

Wraps AVFrame.pict_type

Type:

int

class av.video.frame.PictureType(*values)

Bases: IntEnum

Wraps AVPictureType (AV_PICTURE_TYPE_*).

PictureType Name

Flag Value

Meaning in FFmpeg

NONE

0x0

-

I

0x1

-

P

0x2

-

B

0x3

-

S

0x4

-

SI

0x5

-

SP

0x6

-

BI

0x7

-

Colors

These describe how to interpret the frame’s samples. They are FFmpeg’s raw integer values, and each is named by an enum under Enums. A decoder fills them in from the stream; an encoder passes them through to the container. Setting one relabels the frame, it does not convert the pixels – see VideoFrame.reformat() for that.

VideoFrame.colorspace

The YUV/RGB transformation matrix of the frame, as FFmpeg’s raw integer value. Colorspace names the values.

Wraps AVFrame.colorspace.

Type:

int

VideoFrame.color_range

The signal range of the frame, as FFmpeg’s raw integer value. ColorRange names the values.

Wraps AVFrame.color_range.

Type:

int

VideoFrame.color_trc

The linearization function (a.k.a. transfer characteristic) of the frame, as FFmpeg’s raw integer value. ColorTrc names the values.

Wraps AVFrame.color_trc.

Type:

int

VideoFrame.color_primaries

The RGB/XYZ matrix of the frame, as FFmpeg’s raw integer value. ColorPrimaries names the values.

Wraps AVFrame.color_primaries.

Type:

int

VideoFrame.chroma_location

The position of the chroma samples relative to the luma samples, as FFmpeg’s raw integer value. ChromaLocation names the values.

Wraps AVFrame.chroma_location. FFmpeg spells the same field chroma_sample_location on a codec context, and so does PyAV: see VideoCodecContext.chroma_sample_location.

Type:

int

The matching VideoCodecContext attributes carry the same values for a whole stream. FFmpeg spells the chroma one chroma_location on a frame and chroma_sample_location on a codec context, and PyAV mirrors each C field name, so VideoCodecContext.chroma_sample_location is the codec context spelling of VideoFrame.chroma_location.

Conversions

VideoFrame.reformat(width=None, height=None, format=None, src_colorspace=None, dst_colorspace=None, interpolation=None, threads=None)

Create a new VideoFrame with the given width/height/format/colorspace.

See also

VideoReformatter.reformat() for arguments.

VideoFrame.to_rgb(**kwargs)

Get an RGB version of this frame.

Any **kwargs are passed to VideoReformatter.reformat().

>>> frame = VideoFrame(1920, 1080)
>>> frame.format.name
'yuv420p'
>>> frame.to_rgb().format.name
'rgb24'
VideoFrame.to_image(**kwargs)

Get an RGB PIL.Image of this frame.

Any **kwargs are passed to VideoReformatter.reformat().

Note

PIL or Pillow must be installed.

VideoFrame.to_ndarray(channel_last=False, **kwargs)

Get a numpy array of this frame.

Any **kwargs are passed to VideoReformatter.reformat().

The array returned is generally of dimension (height, width, channels).

Parameters:

channel_last (bool) – If True, the shape of array will be (height, width, channels) rather than (channels, height, width) for the “yuv444p” and “yuvj444p” formats.

Note

Numpy must be installed.

Note

For formats which return an array of uint16, float16 or float32, the samples will be in the system’s native byte order.

Note

For pal8, an (image, palette) tuple will be returned, with the palette being in ARGB (PyAV will swap bytes if needed).

Note

For gbrp formats, channels are flipped to RGB order.

static VideoFrame.from_image(img)

Construct a frame from a PIL.Image.

static VideoFrame.from_ndarray(array, format='rgb24', channel_last=False)

Construct a frame from a numpy array.

Parameters:

channel_last (bool) – If False (default), the shape for the yuv444p and yuvj444p is given by (channels, height, width) rather than (height, width, channels).

Note

For formats which expect an array of uint16, float16 or float32, the samples must be in the system’s native byte order.

Note

for pal8, an (image, palette) pair must be passed. palette must have shape (256, 4) and is given in ARGB format (PyAV will swap bytes if needed).

Note

for gbrp formats, channels are assumed to be given in RGB order.

static VideoFrame.from_dlpack(planes, str format: str = 'nv12', int width: int = 0, int height: int = 0, stream=None, int device_id: int | None = None, bool primary_ctx: bool | None = None, cuda_context=None, bool current_ctx: bool | None = None)

Video Planes

class av.video.plane.VideoPlane

Bases: Plane

buffer_size
height
line_size

Bytes per horizontal line in this plane.

Type:

int

width

Video Reformatters

class av.video.reformatter.VideoReformatter

Bases: object

An object for reformatting size and pixel format of VideoFrame.

It is most efficient to have a reformatter object for each set of parameters you will use as calling reformat() will reconfigure the internal object.

reformat(VideoFrame frame: VideoFrame, width=None, height=None, format=None, src_colorspace=None, dst_colorspace=None, interpolation=None, src_color_range=None, dst_color_range=None, dst_color_trc=None, dst_color_primaries=None, threads=None)

Create a new VideoFrame with the given width/height/format/colorspace.

Returns the same frame untouched if nothing needs to be done to it.

Parameters:

  • width (int) – New width, or None for the same width.

  • height (int) – New height, or None for the same height.

  • format (VideoFormat or str) – New format, or None for the same format.

  • src_colorspace (Colorspace or str) – Current colorspace, or None for the frame colorspace.

  • dst_colorspace (Colorspace or str) – Desired colorspace, or None for the frame colorspace.

  • interpolation (Interpolation or str) – The scaling algorithm to use, or None for BILINEAR. Option flags such as ACCURATE_RND or BITEXACT may be combined with the algorithm using |, e.g. Interpolation.BILINEAR | Interpolation.ACCURATE_RND.

  • src_color_range (ColorRange or str) – Current color range, or None for the UNSPECIFIED.

  • dst_color_range (ColorRange or str) – Desired color range, or None for the UNSPECIFIED.

  • dst_color_trc (ColorTrc or int) – Desired transfer characteristic to tag on the output frame, or None to preserve the source frame’s value. This sets frame metadata only; it does not perform a pixel-level transfer function conversion.

  • dst_color_primaries (ColorPrimaries or int) – Desired color primaries to tag on the output frame, or None to preserve the source frame’s value.

  • threads (int) – How many threads to use for scaling, or 0 for automatic selection based on the number of available CPUs. Defaults to 0 (auto).

Enums

class av.video.reformatter.Interpolation(*values)

Bases: IntFlag

Wraps the SWS_* flags.

Interpolation Name

Flag Value

Meaning in FFmpeg

FAST_BILINEAR

0x1

Fast bilinear

BILINEAR

0x2

Bilinear

BICUBIC

0x4

2-tap cubic B-spline

X

0x8

Experimental

POINT

0x10

Nearest neighbor / point

AREA

0x20

Area averaging

BICUBLIN

0x40

Bicubic luma / Bilinear chroma

GAUSS

0x80

Gaussian approximation

SINC

0x100

Unwindowed Sinc

LANCZOS

0x200

3-tap sinc/sinc

SPLINE

0x400

Unwindowed natural cubic spline

STRICT

0x800

Error out on underspecified conversions

PRINT_INFO

0x1000

Emit verbose scaler info to the log

FULL_CHR_H_INT

0x2000

Full chroma interpolation

FULL_CHR_H_INP

0x4000

Full chroma input

DIRECT_BGR

0x8000

Direct BGR

ACCURATE_RND

0x40000

Accurate rounding

BITEXACT

0x80000

Bit-exact output

UNSTABLE

0x100000

Prefer experimental code paths (testing only)

ERROR_DIFFUSION

0x800000

Error diffusion dither
class av.video.reformatter.Colorspace(*values)

Bases: IntEnum

Wraps the SWS_CS_* flags. There is a bit of overlap in these names which comes from FFmpeg and backwards compatibility.

Colorspace Name

Flag Value

Meaning in FFmpeg

ITU709

0x1

-

FCC

0x4

-

ITU601

0x5

-

ITU624

0x5

-

SMPTE170M

0x5

-

SMPTE240M

0x7

-

DEFAULT

0x5

-

BT2020

0x9

-
class av.video.reformatter.ColorRange(*values)

Bases: IntEnum

Wraps the AVCOL_RANGE_* flags.

ColorRange Name

Flag Value

Meaning in FFmpeg

UNSPECIFIED

0x0

Unspecified

MPEG

0x1

MPEG (limited) YUV range, 219*2^(n-8)

JPEG

0x2

JPEG (full) YUV range, 2^n-1

NB

0x3

Not part of ABI
class av.video.reformatter.ColorTrc(*values)

Bases: IntEnum

Transfer characteristic (gamma curve) of a video frame.

Maps to FFmpeg’s AVColorTransferCharacteristic.

Wraps the AVCOL_TRC_* flags.

ColorTrc Name

Flag Value

Meaning in FFmpeg

BT709

0x1

BT.709

UNSPECIFIED

0x2

Unspecified

GAMMA22

0x4

Gamma 2.2 (BT.470M)

GAMMA28

0x5

Gamma 2.8 (BT.470BG)

SMPTE170M

0x6

SMPTE 170M

SMPTE240M

0x7

SMPTE 240M

LINEAR

0x8

Linear

LOG

0x9

Logarithmic (100:1 range)

LOG_SQRT

0xA

Logarithmic (100*sqrt(10):1 range)

IEC61966_2_4

0xB

IEC 61966-2-4 (sRGB)

BT1361_ECG

0xC

BT.1361 extended colour gamut

IEC61966_2_1

0xD

IEC 61966-2-1 (sYCC)

BT2020_10

0xE

BT.2020 10-bit

BT2020_12

0xF

BT.2020 12-bit

SMPTE2084

0x10

SMPTE 2084 (PQ, HDR10)

SMPTE428

0x11

SMPTE 428-1

ARIB_STD_B67

0x12

ARIB STD-B67 (HLG)

V_LOG

0x100

Panasonic V-Log (not part of H.273)
class av.video.reformatter.ColorPrimaries(*values)

Bases: IntEnum

Color primaries of a video frame.

Maps to FFmpeg’s AVColorPrimaries.

Wraps the AVCOL_PRI_* flags.

ColorPrimaries Name

Flag Value

Meaning in FFmpeg

BT709

0x1

BT.709 / sRGB / sYCC

UNSPECIFIED

0x2

Unspecified

BT470M

0x4

BT.470M

BT470BG

0x5

BT.470BG / BT.601-6 625

SMPTE170M

0x6

SMPTE 170M / BT.601-6 525

SMPTE240M

0x7

SMPTE 240M

FILM

0x8

Generic film (Illuminant C)

BT2020

0x9

BT.2020 / BT.2100

SMPTE428

0xA

SMPTE 428-1 / XYZ

SMPTE431

0xB

SMPTE 431-2 (DCI-P3)

SMPTE432

0xC

SMPTE 432-1 (Display P3)

EBU3213

0x16

EBU 3213-E / JEDEC P22

V_GAMUT

0x100

Panasonic V-Gamut (not part of H.273)
class av.video.reformatter.ChromaLocation(*values)

Bases: IntEnum

Location of the chroma samples relative to the luma samples.

Maps to FFmpeg’s AVChromaLocation.

Wraps the AVCHROMA_LOC_* flags.

ChromaLocation Name

Flag Value

Meaning in FFmpeg

UNSPECIFIED

0x0

Unspecified

LEFT

0x1

MPEG-2/4 4:2:0, H.264 default for 4:2:0

CENTER

0x2

MPEG-1 4:2:0, JPEG 4:2:0, H.263 4:2:0

TOPLEFT

0x3

ITU-R 601, SMPTE 274M/296M, MPEG-2 4:2:2

TOP

0x4

Top

BOTTOMLEFT

0x5

Bottom left

BOTTOM

0x6

Bottom