⚙️ Configuration

Tabby server will look for a configuration file at ~/.tabby/config.toml for advanced features.

Repository context for code completion

To enable repository level context for code completion, you can add the following to your configuration file:

~/.tabby/config.toml

# Index three repositories' source code as additional context for code completion.

[[repositories]]
name = "tabby"
git_url = "https://github.com/TabbyML/tabby.git"

# git through ssh protocol.
[[repositories]]
name = "CTranslate2"
git_url = "git@github.com:OpenNMT/CTranslate2.git"

# local directory is also supported!
[[repositories]]
name = "repository_a"
git_url = "file:///home/users/repository_a"

Once this is set, you can run tabby scheduler to index the source code repository.

tip

By default, tabby scheduler runs in a daemon and processes its pipeline every 5 hours. To run the pipeline immediately, use tabby scheduler --now.

artifacts produced by tabby scheduler

~/.tabby % ls dataset
data.jsonl

~/.tabby % ls index
1a8729fa34d844df984b444f4def1456.fast      2ed712d4a7a44ed797dd4ff5ceaf4312.fieldnorm
b42ca53fe6f94d0c8e96f947318278ba.idx       1a8729fa34d844df984b444f4def1456.fieldnorm 
2ed712d4a7a44ed797dd4ff5ceaf4312.idx       b42ca53fe6f94d0c8e96f947318278ba.pos
...

In a code completion request, additional context from the source code repository will be attached to the prompt for better completion quality. For example:

Example prompt for code completion, with retrieval augmented enabled

// Path: crates/tabby/src/serve/engine.rs
// fn create_llama_engine(model_dir: &ModelDir) -> Box<dyn TextGeneration> {
//     let options = llama_cpp_bindings::LlamaEngineOptionsBuilder::default()
//         .model_path(model_dir.ggml_q8_0_file())
//         .tokenizer_path(model_dir.tokenizer_file())
//         .build()
//         .unwrap();
//
//     Box::new(llama_cpp_bindings::LlamaEngine::create(options))
// }
//
// Path: crates/tabby/src/serve/engine.rs
// create_local_engine(args, &model_dir, &metadata)
//
// Path: crates/tabby/src/serve/health.rs
// args.device.to_string()
//
// Path: crates/tabby/src/serve/mod.rs
// download_model(&args.model, &args.device)
    } else {
        create_llama_engine(model_dir)
    }
}

fn create_ctranslate2_engine(
    args: &crate::serve::ServeArgs,
    model_dir: &ModelDir,
    metadata: &Metadata,
) -> Box<dyn TextGeneration> {
    let device = format!("{}", args.device);
    let options = CTranslate2EngineOptionsBuilder::default()
        .model_path(model_dir.ctranslate2_dir())
        .tokenizer_path(model_dir.tokenizer_file())
        .device(device)
        .model_type(metadata.auto_model.clone())
        .device_indices(args.device_indices.clone())
        .build()
        .⮹

Usage Collection

Tabby collects usage stats by default. This data will only be used by the Tabby team to improve its services.

What data is collected?

We collect non-sensitive data that helps us understand how Tabby is used. For now we collects serve command you used to start the server. As of the date 04/18/2024, the following information has been collected:

struct HealthState {
    model: String,
    chat_model: Option<String>,
    device: String,
    arch: String,
    cpu_info: String,
    cpu_count: usize,
    cuda_devices: Vec<String>,
    version: Version,
    webserver: Option<bool>,
}

For an up-to-date list of the fields we have collected, please refer to health.rs.

How to disable it

To disable usage collection, set the TABBY_DISABLE_USAGE_COLLECTION environment variable by export TABBY_DISABLE_USAGE_COLLECTION=1.