Laravel Related Content: Semantic Relationships Using pgvector

Finding related content in a Laravel application often involves building custom queries or relying on keyword matching, which can miss important semantic connections. To address this issue, the Laravel Related Content package, developed by Vladislav Stoitsov, leverages vector embeddings and PostgreSQL’s pgvector extension to automatically discover and link related content across different model types based on meaning rather than mere keywords. As of the time of this writing, the package is currently in beta.

Main Features

The Laravel Related Content package offers several key features that enhance the way related content is managed and retrieved:

• Pre-computed Related Links: Related content is calculated when saving a model, rather than on every page load, which optimizes performance.
• Fast Lookups: The package allows for O(1) relationship queries, making it significantly faster than real-time similarity searches.
• Cross-Model Relationships: It enables finding related content across different model types, such as linking an article to related events or community links.
• Multiple Embedding Providers: The package supports various embedding providers, including OpenAI and Ollama.
• Queue Support: Embeddings can be processed in the background, improving the efficiency of content management.
• Semantic Search: The package allows for searching content based on meaning, not just keywords, enhancing the relevance of search results.

Getting Started

To use the Laravel Related Content package, you need to have PostgreSQL with the pgvector extension enabled. If you haven’t done this yet, you can run the following command in your database:

CREATE EXTENSION IF NOT EXISTS vector;

After installing the package and running its migrations, you will need to configure your embedding provider in the .env file. Here’s how to set it up for both OpenAI and a local Ollama instance:

For OpenAI:

RELATED_CONTENT_PROVIDER=openai
OPENAI_API_KEY=your-api-key
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
OPENAI_EMBEDDING_DIMENSIONS=1536

For a Local Ollama Instance:

RELATED_CONTENT_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
OLLAMA_EMBEDDING_DIMENSIONS=768

Adding Related Content to Models

To integrate related content functionality into your models, you need to add the HasRelatedContent trait to any model you wish to enhance. Additionally, you will need to define which fields should be used for generating embeddings. Here’s an example using an Article model:

class Article extends Model {
    use HasRelatedContent;

    public function embeddableFields(): array {
        return ['title', 'excerpt', 'content'];
    }
}

When a model is saved, the package generates an embedding from the specified fields and finds related content across all configured models.

Retrieving Related Content

Retrieving related models is straightforward with the Laravel Related Content package. You can fetch related models using a single method call:

// Get all related models
$related = $article->getRelatedModels();

// Limit the number of results
$related = $article->getRelatedModels(5);

// Get related models of a specific type
$events = $article->getRelatedOfType(Event::class);

Since these relationships are pre-computed, the calls made to retrieve related content are standard database queries, which means they execute quickly without additional API requests.

Cross-Model Relationships

One of the standout features of the Laravel Related Content package is its ability to create links between different models. To enable cross-model relationships, you need to register the models that should participate in these relationships in the config/related-content.php file. The package will then automatically find connections between them.

'models' => [
    AppModelsArticle::class,
    AppModelsEvent::class,
    AppModelsCommunityLink::class,
],

For example, an article discussing “Laravel testing” could surface a related conference event or community link on the same topic. You can also semantically search across all configured models based on meaning rather than exact keywords:

$service = app(RelatedContentService::class);
$results = $service->search('Laravel AI tools and best practices');

This search will return matching content from any registered model type based on the semantic similarity of the query to stored embeddings.

Rebuilding Embeddings

If you need to generate embeddings for existing records or regenerate them after changing embeddable fields, the package includes an Artisan command to facilitate this process:

• Process models that are missing embeddings: php artisan related-content:rebuild
• Process a specific model (missing only): php artisan related-content:rebuild "AppModelsArticle"
• Regenerate all embeddings: php artisan related-content:rebuild --force
• Process synchronously instead of queuing: php artisan related-content:rebuild --sync

Events

The Laravel Related Content package also dispatches a RelatedContentSynced event whenever relationships are updated. This feature allows developers to trigger additional logic, such as cache invalidation or notifications, ensuring that your application remains responsive and up-to-date.

Conclusion

The Laravel Related Content package, utilizing PostgreSQL’s pgvector extension, provides a powerful tool for managing semantic relationships in your Laravel applications. By leveraging vector embeddings, it enhances the way related content is discovered and linked, moving beyond traditional keyword matching. This package not only improves performance through pre-computed relationships but also offers flexibility in searching and linking various model types.

Note: The Laravel Related Content package is currently in beta, and developers are encouraged to explore its capabilities and provide feedback to improve its functionality.

Frequently Asked Questions