modelscope-scepter/docs/en/scepter/transforms.md

Transforms

Data pre-processing module

Overview

Supports various data pre-processing methods：

1. scepter.transforms.image
2. scepter.transforms.io
3. scepter.transforms.io_video
4. scepter.transforms.tensor
5. scepter.transforms.augmention
6. scepter.transforms.video
7. scepter.transforms.transform_xl
8. scepter.transforms.identity
9. scepter.transforms.compose

---

Basic Usage

# 以 scepter.modules.transform.image.RandomResizedCrop 为例
from scepter.modules.transform.image import RandomResizedCrop
from scepter.modules.utils.config import Config
import PIL

cfg = Config(load=False,
             cfg_dict={"SIZE": 224, "RATIO": [3. / 4., 4. / 3.], "SCALE": [0.08, 1.0], "INTERPOLATION": "bilinear"})
transform = RandomResizedCrop(cfg)
input_img = {"img": PIL.Image}
output_img = transform(input_img)

---

scepter.modules.transform.image

Some pre-processing methods used for images.

scepter.modules.transform.image.ImageTransform

Initialize the ImageTransform class, obtain and define some necessary parameters from cfg.

Parameters

• INPUT_KEY —— (str) input key or key list
• OUTPUT_KEY —— (str) output key or key list
• BACKEND —— (str) backend, choose from pillow, cv2, torchvision

scepter.modules.transform.image.RandomResizedCrop

Randomly crop the image to a specified size.

Parameters

• SIZE —— (int) crop size
• RATIO —— (list) ratio
• SCALE —— (list) scale
• INTERPOLATION —— (str) interpolation

scepter.modules.transform.image.RandomHorizontalFlip

Randomly horizontally flip the given image with a given probability(P).

Parameters

• P —— (float) probability

scepter.modules.transform.image.Normalize

Normalize the image using mean and standard deviation(std).

Parameters

• MEAN —— (list) mean
• STD —— (list) std

scepter.modules.transform.image.ImageToTensor

transform PIL.Image / numpy.ndarray / unit8 to float32 tensor.

Parameters

scepter.modules.transform.image.Resize

Resize the given image to the given Size.

Parameters

• INTERPOLATION —— (str) interpolation
• SIZE —— (int) resized size

scepter.modules.transform.image.CenterCrop

Crop the given image from the center.

Parameters

• SIZE —— (int) crop size

scepter.modules.transform.image.FlexibleResize

Resize the given image to the given Size.

Parameters

• INTERPOLATION —— (str) interpolation

scepter.modules.transform.image.FlexibleCenterCrop

Center crop the given image to the given Size.

Parameters

scepter.modules.transform.io

Some methods for reading images from local disk.

scepter.modules.transform.io.LoadPILImageFromFile

Read a local image file into PIL.Image format.

Parameters

• RGB_ORDER —— (str) "RGB" or "BGR"

scepter.modules.transform.io.LoadCvImageFromFile

Read a local image file into cv2 format.

Parameters

• RGB_ORDER —— (str) "RGB" or "BGR"

scepter.modules.transform.io.LoadImageFromFile

Read a local image file into a specific format.

Parameters

• RGB_ORDER —— (str) "RGB" or "BGR"
• BACKEND —— (str) "pillow" or "cv2" or "torchvision"

scepter.modules.transform.io.LoadImageFromFileList

Read a set of input images into a specific format.

Parameters

• RGB_ORDER —— (str) "RGB" or "BGR"
• BACKEND —— (str) "pillow" or "cv2" or "torchvision"
• FILE_KEYS —— (list) The file keys for input

scepter.modules.transform.io_video

Some methods for reading videos from local disk.

scepter.modules.transform.io_video.DecodeVideoToTensor

Decode local video files into tensors.

Parameters

• NUM_FRAMES —— (int) decode frames number
• TARGET_FPS —— (int) decode frames fps, default is 30.
• SAMPLE_MODE —— (str) interval or segment sampling, default is interval
• SAMPLE_INTERVAL —— (int) sample interval between output frames for interval sample mode
• SAMPLE_MINUS_INTERVAL —— (float) wheather minus interval for interval sample mode
• REPEAT —— (str) number of clips to be decoded from each video

scepter.modules.transform.io_video.LoadVideoFromFile

Decode and read local video files into a sequence of frames.

Parameters

• NUM_FRAMES —— (int) decode frames number
• SAMPLE_TYPE —— (str) sample type
• CLIP_DURATION —— (float) needed for 'interval' sampling type
• DECODER —— (str) video decoder name

scepter.modules.transform.tensor

Some methods for processing tensors.

scepter.modules.transform.tensor.ToTensor

Convert input data from other formats into tensors.

Parameters

• KEYS —— (list) keys of input data

scepter.modules.transform.tensor.Select

Select some keys from the input data and output them.

Parameters

• META_KEYS —— (list) chosen keys of input data

scepter.modules.transform.tensor.Rename

Rename the keys of the input data.

Parameters

• IN_KEYS —— (list) input data keys
• OUT_KEYS —— (list) output data keys

scepter.modules.transform.augmention

Some methods for enhancing the colors in images.

scepter.modules.transform.augmention.ColorJitterGeneral

Randomly adjust the brightness, contrast, and saturation of an image.

Parameters

• BRIGHTNESS —— (float) (float or tuple of float (min, max)): How much to jitter brightness
• CONTRAST —— (float) (float or tuple of float (min, max)): How much to jitter contrast
• SATURATION —— (float) (float or tuple of float (min, max)): How much to jitter saturation
• HUE —— (float) (float or tuple of float (min, max)): How much to jitter hue
• GRAYSCALE —— (float) probablitities for rgb-to-gray 0~1
• CONSISTENT —— (bool) for video input whether the augment scale is consistent or not
• SHUFFLE —— (bool) shuffle the transform's order when there are multiple transforms
• GRAY_FIRST —— (bool) whether use grayscale or not
• IS_SPLIT —— (bool) whether randomly chance the channel as the gray results

scepter.modules.transform.video

Some methods for processing videos.

scepter.modules.transform.video.VideoTransform

To initialize a VideoTransform class and define the necessary parameters from a cfg.

Parameters

• INPUT_KEY —— (str) input key or key list
• OUTPUT_KEY —— (str) output key or key list
• BACKEND —— (str) backend, choose from pillow, cv2, torchvision

scepter.modules.transform.video.RandomResizedCropVideo

Perform random cropping of the video to a specified size.

Parameters

• META_KEYS —— (list) chosen keys of input data

scepter.modules.transform.video.CenterCropVideo

Renaming the keys of the input data.

Parameters

• SIZE —— (int) crop size
• RATIO —— (list) ratio
• SCALE —— (list) scale
• INTERPOLATION —— (str) interpolation

scepter.modules.transform.video.RandomHorizontalFlipVideo

Randomly flip the given video horizontally with a given probability.

Parameters

• P —— (float) probability

scepter.modules.transform.video.NormalizeVideo

Normalize the video using the mean and standard deviation(std).

Parameters

• MEAN —— (list) mean
• STD —— (list) std

scepter.modules.transform.video.VideoToTensor

transform PIL.Image / numpy.ndarray / unit8 to float32 tensor.

Parameters

scepter.modules.transform.video.AutoResizedCropVideo

Crop the given video from the center.

Parameters

• SCALE —— (list) scale

scepter.modules.transform.video.ResizeVideo

Resize the given video to the specified dimensions.

Parameters

• SCALE —— (list) scale
• INTERPOLATION —— (str) interpolation

scepter.modules.transform.transform_xl

Some methods for image processing in SDXL to obtain the desired coordinates.

scepter.modules.transform.transform_xl.FlexibleCropXL

Crop an image and obtain its original size, target size, and cropping coordinates (top/left).

Parameters

• INPUT_KEY —— (str) input key or key list
• OUTPUT_KEY —— (str) output key or key list
• BACKEND —— (str) backend, choose from pillow, cv2, torchvision
• SIZE —— (int or list) crop size if 'image_size' not in meta

scepter.modules.transform.identity

Some methods to process images and obtain the required coordinates in SDXL.

scepter.modules.transform.identity.Identity

Return the image itself.

Parameters

scepter.modules.transform.compose

Combine various transform methods.

scepter.modules.transform.compose.Compose

Combine the various transform objects from scepter.transforms into a pipeline.

Parameters

• TRANSFORMS —— (list) transform config list