SafetensorsQLORAGenerator

Guide & worked example · Source

SafetensorsQLORAGenerator(
    model_path,
    tokenizer_path=None,
    examples=32,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    load_in_4bit=False,
    max_attempts=20,
    rows_per_prompt=None,
    progress_path=None,
    device=None,
)

QLoRA-fine-tuned LLM row generator.

The benchmark calls fit_adapter explicitly because fine-tuning can be expensive and model-specific. After an adapter exists, sampling follows the same CSV parsing path as the in-context generator.

Attributes

Name	Description
base_model_path
device
do_sample
examples
load_in_4bit
max_attempts
max_new_tokens
model_path
params
progress_path
rows_per_prompt
temperature
tokenizer_path

Methods

Name	Description
fit
fit_adapter
sample
summary	Print a summary of the model results.
to	Move all model parameters to specified device.

fit

SafetensorsQLORAGenerator.fit(X, column_names=None)

fit_adapter

SafetensorsQLORAGenerator.fit_adapter(
    output_dir,
    num_train_epochs=1.0,
    learning_rate=0.0002,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=8,
    rows_per_completion=8,
    train_samples=None,
    max_length=1024,
    lora_r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    seed=123,
)

sample

SafetensorsQLORAGenerator.sample(n)

summary

SafetensorsQLORAGenerator.summary()

Print a summary of the model results.

to

SafetensorsQLORAGenerator.to(device)

Move all model parameters to specified device.

Parameters

Name	Type	Description	Default
device	torch.device or str	Target device (‘cuda’, ‘cpu’, etc.)	required

Returns

Name	Type	Description
	BaseEstimator	Self for method chaining