Importer Implementation Guide

Overview

The Inly Core framework provides a robust synchronization architecture for importing and exporting data between external services and your internal domain models. The architecture follows clear patterns for Extract-Transform-Load (ETL) operations, maintains separation between domain and external object models, and provides comprehensive logging through Traceflow. This document uses BitsData/Snowflake and HubSpot as examples of external service integrations, but the patterns apply to any external API or data source you want to synchronize with.

Architecture Components

The Inly Core importer architecture follows a hierarchical class-based approach with clear separation of concerns:

ResourceImporter (abstract base)
    ↓
ServiceResourcesImporter (service-specific base)
    ↓
Individual Resource Importers (concrete implementations)

1. Extract-Transform-Load (ETL) Pattern

The system implements a clear ETL pattern with distinct responsibilities:

Extract Phase

External Services: Service classes use REST/GraphQL APIs to fetch data from external systems
Data Sources: Each service handles authentication, pagination, and API-specific concerns
Individual Importers: Each resource type has its own importer class
Examples: BusinessCentral service with ItemImporter, ColorImporter; HubSpot service with ContactImporter, DealImporter

Transform Phase

Resource Importers: Each importer handles transformation for its specific resource type
HasResourceObject Trait: Provides ETL methods for resource models (fillTransformedData, saveTransformedData)
Property Mapping: Map external field names to internal field names within each importer
Batch Timestamping: Track when data was imported and from which batch

Load Phase

Upsert Operations: Individual importers handle inserting or updating records for their resource type
Batch Processing: Support for bulk operations with timestamps via ResourceImporter base class
Relationship Management: Handle complex relationships between entities within specific importers

2. Model Structure

The architecture maintains a clear separation between different types of models:

Domain Models (`app/Models/`)

Domain models represent the internal business entities:

// app/Models/Customer.php
class Customer extends Model
{
    // Internal business logic
    public function getFullNameAttribute(): string
    {
        return $this->first_name . ' ' . $this->last_name;
    }

    // Relationships
    public function orders()
    {
        return $this->hasMany(Order::class);
    }
}

Characteristics:

Use internal field names and business logic
Contain computed properties and relationships
May have external service integration traits
Single source of truth for business data

External Object Models (`app/Models/`)

External object models store raw data from external systems. They should be prefixed with the service name (e.g., HubSpotContact, BusinessCentralItem) and implement ResourceObjectContract using the HasResourceObject trait.

// app/Models/HubSpotContact.php
class HubSpotContact extends Model implements ResourceObjectContract
{
    use HasFactory;
    use HasResourceObject;
    use HasStringKey;

    protected $table = 'hubspot_contacts';

    public function deals(): HasMany
    {
        return $this->hasMany(HubSpotDeal::class, 'hubspot_contact_id');
    }

    public function getObjectTitle(): ?string
    {
        return $this->name ?? $this->email;
    }

    public static function getObjectCollectionTitle(?bool $singular = false): string
    {
        return __('HubSpot Contacts');
    }
}

Characteristics:

Implement ResourceObjectContract for display and ETL capabilities
Use HasResourceObject trait for complete implementation
Store raw JSON data in data field (provided by trait)
Track batch import timestamps (created_from_batch, updated_from_batch)
Track source timestamps (source_created_at, source_updated_at)
Use external system’s primary keys with HasStringKey trait
Preserve original data structure for debugging
Can have URLs pointing to external systems (e.g., “view in HubSpot”)
No activity logging (only domain objects have that)

3. Importer Architecture

The new importer architecture uses a three-layer approach:

Layer 1: ResourceImporter (Abstract Base)

The foundation class that provides common functionality for all importers:

// vendor/inly/core/src/Support/Import/ResourceImporter.php
abstract class ResourceImporter
{
    use HandlesException;

    protected function importWithPaginator(?string $environment, Paginator $paginator, callable $upsert) { /* ... */ }
    protected function importSingle(?string $environment, array $result, callable $upsert) { /* ... */ }

    /**
     * Finds all the importers in the same directory as the current class.
     */
    public static function getAvailableImporters(): array { /* ... */ }
}

Layer 2: Service-Specific Base Importers

Each external service has its own base importer that extends ResourceImporter:

// app/Import/HubSpot/HubSpotResourcesImporter.php
abstract class HubSpotResourcesImporter extends ResourceImporter
{
    abstract protected function upsert($data, array $metadata): Model;
    abstract protected function getList(array|HubSpotSearchFilter $filters = []): OffsetPaginator;
    abstract protected function getSingleResource(string $id, array $filters = []);
    abstract protected function getRequiredProperties(): array;

    public function import(array|HubSpotSearchFilter $filters = []): void
    {
        $this->importWithPaginator(null, $this->getList($filters), fn($data, $metadata) => $this->upsert($data, $metadata));
    }

    public function get(string $id): void
    {
        $this->importSingle(null, $this->getSingleResource($id), fn($data, $metadata) => $this->upsert($data, $metadata));
    }
}

// app/Import/BusinessCentral/BusinessCentralResourcesImporter.php
abstract class BusinessCentralResourcesImporter extends ResourceImporter
{
    abstract protected function getAllPaginated(string $company): OffsetPaginator;
    abstract protected function getById(string $company, string $id);
    abstract protected function upsert(array $data, array $metadata, string $company): Model;

    public function import(string $company): void
    {
        $paginator = $this->getAllPaginated($company);
        $this->importWithPaginator($company, $paginator, fn($data, $metadata) => $this->upsert($data, $metadata, $company));
    }

    public function get(string $company, string $id): void
    {
        $this->importSingle($company, $this->getById($company, $id), fn($data, $metadata) => $this->upsert($data, $metadata, $company));
    }
}

Layer 3: Individual Resource Importers

Each resource type has its own dedicated importer class:

// app/Import/HubSpot/ContactImporter.php
class ContactImporter extends HubSpotResourcesImporter
{
    public function upsert($data, array $metadata): Model
    {
        $hubSpotContact = HubSpotContact::firstOrNew([
            'id' => data_get($data, 'id'),
        ]);

        // Combine firstname and lastname into name
        $name = trim((data_get($data, 'properties.firstname') ?? '') . ' ' . (data_get($data, 'properties.lastname') ?? ''));
        $name = empty($name) ? null : $name;

        $hubSpotContact->fillTransformedData($data, [
            'email' => data_get($data, 'properties.email'),
            'name' => $name,
            'phone' => data_get($data, 'properties.phone'),
            'company' => data_get($data, 'properties.company'),
        ], $metadata)->save();

        return $hubSpotContact;
    }

    protected function getList(array|HubSpotSearchFilter $filters = []): OffsetPaginator
    {
        $properties = $this->getRequiredProperties();

        if (empty($filters)) {
            return $this->getHubSpotClient()->contacts()->paginate($properties);
        } else {
            return $this->getHubSpotClient()->contacts()->paginateSearch($filters, $properties);
        }
    }

    protected function getSingleResource(string $id, array $filters = [])
    {
        return $this->getHubSpotClient()->contacts()->get($id, $this->getRequiredProperties());
    }

    protected function getRequiredProperties(): array
    {
        return [
            'email', 'firstname', 'lastname', 'phone', 'company',
            'jobtitle', 'lifecyclestage', 'leadstatus', 'hubspot_owner_id',
            // ... more properties
        ];
    }
}

// app/Import/BusinessCentral/ItemImporter.php
class ItemImporter extends BusinessCentralResourcesImporter
{
    public function upsert($data, array $metadata, string $company): Model
    {
        $item = BusinessCentralItem::firstOrNew([
            'id' => data_get($data, 'id'),
        ]);

        $item->fillTransformedData($data, [
            'company' => $company,
            'title' => data_get($data, 'displayName'),
        ], $metadata)->save();

        return $item;
    }

    public function getAllPaginated(string $company): OffsetPaginator
    {
        return $this->getClient($company)->items()->paginate();
    }

    public function getById(string $company, string $id)
    {
        return $this->getClient($company)->items()->get($id);
    }

    protected function getClient(string $company): BusinessCentral
    {
        return app(BusinessCentralClientFactory::class)->make(
            BusinessCentralCompany::from($company)
        );
    }
}

4. Service Classes

Service classes handle the low-level communication with external systems. These are implemented using the Saloon SDK pattern as documented in the Saloon SDK Implementation Guide. Key responsibilities:

Managing connections and authentication
Handling API calls or database queries
Managing pagination and rate limiting
Providing consistent interfaces for data retrieval

Examples: BusinessCentral, HubSpot, Harvest, Notion services For detailed implementation of service classes, refer to the Saloon SDK Implementation Guide.

5. Command Implementation

Master Import Command

The framework provides a master command pattern at stubs/laravel/app/Console/Commands/ImportResourceCommand.php:

// app/Console/Commands/ImportResourceCommand.php
class ImportResourceCommand extends BaseImportResourceCommand
{
    protected array $commands = [
        'ExternalService' => ImportExternalServiceResource::class,
        'HubSpot' => ImportHubSpotResource::class,
        'BusinessCentral' => ImportBusinessCentralResource::class,
    ];
}

Service-Specific Import Commands

The new architecture uses the getAvailableImporters() method to dynamically discover importer classes:

// app/Console/Commands/ImportHubSpotResource.php
class ImportHubSpotResource extends BaseCommand
{
    protected $signature = 'hubspot:import-resource
                            {resource? : The resource type to import}
                            {--id= : The HubSpot ID to import}
                            {--updated-minutes-ago= : Filter by resources updated in the last X minutes}
                            {--silent : Run in silent mode (no output)}';

    protected $description = 'Import resources from HubSpot';

    public function handle()
    {
        $importer = $this->argument('resource') ?? (! $this->option('silent') ? select(
            'Select a resource to import',
            collect(HubSpotResourcesImporter::getAvailableImporters())->mapWithKeys(function (string $importer) {
                return [
                    $importer => class_basename($importer),
                ];
            })->all()
        ) : null);

        if (! $importer) {
            $this->error('Resource argument is required when using --silent flag');
            return 1;
        }

        $id = $this->option('id') ?? (! $this->option('silent') ? text(
            'Enter an ID to import specific resource',
            placeholder: 'Press enter to import all resources'
        ) : null);

        $updatedAt = $this->option('updated-minutes-ago');
        $updatedAt = $updatedAt ? now()->subMinutes($updatedAt) : null;

        $importer = app($importer);
        $filters = HubSpotSearchFilter::make();

        if ($updatedAt) {
            $filters->whereGreaterThan(HubSpotResourcesImporter::getUpdatedAtPropertyName($importer), $updatedAt->toISOString());
        }

        try {
            // Execute the import
            if ($id) {
                $importer->get($id);
            } else {
                $importer->import($filters);
            }

            if (! $this->option('silent')) {
                $this->info('Import completed successfully!');
            }
        } catch (Exception $e) {
            $this->error('Import failed: ' . $e->getMessage());
            return 1;
        }

        return 0;
    }
}

// app/Console/Commands/ImportBusinessCentralResource.php
class ImportBusinessCentralResource extends BaseCommand
{
    protected $signature = 'businesscentral:import-resource
                            {resource? : The resource type to import}
                            {--company= : The company to import from (sweden, england, usa)}
                            {--id= : The BusinessCentral ID to import}
                            {--silent : Run in silent mode (no output)}';

    protected $description = 'Import resources from BusinessCentral';

    public function handle()
    {
        $importer = $this->argument('resource') ?? (! $this->option('silent') ? select(
            'Select a resource to import',
            collect(BusinessCentralResourcesImporter::getAvailableImporters())->mapWithKeys(function (string $importer) {
                return [
                    $importer => class_basename($importer),
                ];
            })->all()
        ) : null);

        if (! $importer) {
            $this->error('Resource argument is required when using --silent flag');
            return 1;
        }

        $company = $this->option('company') ?? (! $this->option('silent') ? select(
            'Select a company to import from',
            collect(BusinessCentralCompany::cases())->map(fn (BusinessCentralCompany $company) => $company->value)->all(),
            'sweden'
        ) : null);

        $id = $this->option('id') ?? (! $this->option('silent') ? text(
            'Enter an ID to import specific resource',
            placeholder: 'Press enter to import all resources'
        ) : null);

        $importer = app($importer);

        try {
            // Execute the import
            if ($id) {
                $importer->get($company, $id);
            } else {
                $importer->import($company);
            }

            if (! $this->option('silent')) {
                $this->info('Import completed successfully!');
            }
        } catch (Exception $e) {
            $this->error('Import failed: ' . $e->getMessage());
            return 1;
        }

        return 0;
    }
}

Usage Examples

# Import from specific service (interactive)
php artisan hubspot:import-resource
php artisan businesscentral:import-resource

# Import specific resource by class name
php artisan hubspot:import-resource "App\Import\HubSpot\ContactImporter"
php artisan businesscentral:import-resource "App\Import\BusinessCentral\ItemImporter" --company=sweden

# Import specific resource by ID
php artisan hubspot:import-resource "App\Import\HubSpot\ContactImporter" --id=12345
php artisan businesscentral:import-resource "App\Import\BusinessCentral\ItemImporter" --company=sweden --id=abc123

# Import recent updates only
php artisan hubspot:import-resource "App\Import\HubSpot\ContactImporter" --updated-minutes-ago=60

# Silent mode for scheduled tasks
php artisan hubspot:import-resource "App\Import\HubSpot\ContactImporter" --silent
php artisan businesscentral:import-resource "App\Import\BusinessCentral\ItemImporter" --company=sweden --silent

6. Workflow Integration

The new importer architecture integrates seamlessly with Laravel workflows for automated import processes:

// app/Workflows/ImportHubSpotWorkflow.php
#[Dispatchable]
class ImportHubSpotWorkflow extends Workflow
{
    public static $category = 'HubSpot';

    public function execute(HubSpotResourcesImporter $importer, ?Carbon $updatedAt = null)
    {
        yield ActivityStub::make(ImportHubSpotActivity::class, $importer, $updatedAt);
    }
}

// app/Workflows/Activities/ImportHubSpotActivity.php
class ImportHubSpotActivity extends Activity
{
    public function execute(HubSpotResourcesImporter $importer, ?Carbon $updatedAt = null)
    {
        traceflow()->info('Starting HubSpot import: ' . get_class($importer));

        $filters = HubSpotSearchFilter::make();
        if ($updatedAt) {
            $filters->whereGreaterThan(
                HubSpotResourcesImporter::getUpdatedAtPropertyName($importer),
                $updatedAt->toISOString()
            );
        }

        $importer->import($filters);
    }
}

This allows you to orchestrate complex import processes, chain multiple importers, and handle dependencies between different resources.

7. Traceflow Logging System

Traceflow provides unified logging across the entire sync process:

// Basic usage in your importer
traceflow()->info('Starting external service import of resource ' . $type);
traceflow()->error('Error importing resource', $exception);
traceflow()->success('Import completed successfully');

// With additional context
traceflow()->info('Processing batch', [
    'batch_size' => count($items),
    'resource_type' => $type,
    'timestamp' => now(),
]);

Key Features:

Contextual Logging: Automatically includes context information
Multiple Targets: Database storage, console output, monitoring systems
Structured Data: Supports additional context data
Error Tracking: Automatic exception logging with stack traces

8. Implementation Guide for New Services

Follow these steps to implement a new external service importer using the new architecture:

Step 1: Create Service Class

Create your service class following the Saloon SDK Implementation Guide. The service must provide methods for fetching resources that return pagination and data compatible with your resource classes.

Step 2: Create External Object Models

Create models implementing ResourceObjectContract with the HasResourceObject trait:

// app/Models/NewServiceContact.php
class NewServiceContact extends Model implements ResourceObjectContract
{
    use HasFactory;
    use HasResourceObject;
    use HasStringKey;

    protected $table = 'new_service_contacts';

    public function getObjectTitle(): ?string
    {
        return $this->name ?? $this->email;
    }

    public static function getObjectCollectionTitle(?bool $singular = false): string
    {
        return __('NewService Contacts');
    }

    // Add relationships as needed
    public function orders(): HasMany
    {
        return $this->hasMany(Order::class, 'new_service_contact_id');
    }
}

Step 3: Create Service-Specific Base Importer

Create a base importer class for your service. Environment can be null, or be used to support multiple instances of the same service. E.g. syncing orders from multiple Shopify instances.

// app/Import/NewService/NewServiceResourcesImporter.php
abstract class NewServiceResourcesImporter extends ResourceImporter
{
    abstract protected function upsert($data, array $metadata): Model;
    abstract protected function getList(array $filters = []): OffsetPaginator;
    abstract protected function getSingleResource(string $id): array;

    public function import($environment, array $filters = []): void
    {
        $this->importWithPaginator($environment, $this->getList($filters), fn($data, $metadata) => $this->upsert($data, $metadata));
    }

    public function get($environment, string $id): void
    {
        $this->importSingle($environment, $this->getSingleResource($id), fn($data, $metadata) => $this->upsert($data, $metadata));
    }

    protected function getNewServiceClient(): NewService
    {
        return app(NewService::class);
    }
}

Step 4: Create Individual Resource Importers

Create specific importer classes for each resource type:

// app/Import/NewService/ContactImporter.php
class ContactImporter extends NewServiceResourcesImporter
{
    public function upsert($data, array $metadata): Model
    {
        $contact = NewServiceContact::firstOrNew([
            'id' => data_get($data, 'id'),
        ]);

        $contact->fillTransformedData($data, [
            'name' => data_get($data, 'name'),
            'email' => data_get($data, 'email'),
            'phone' => data_get($data, 'phone'),
            // ... other transformed fields
        ], $metadata)->save();

        return $contact;
    }

    protected function getList(array $filters = []): OffsetPaginator
    {
        return $this->getNewServiceClient()->contacts()->paginate($filters);
    }

    protected function getSingleResource(string $id): array
    {
        return $this->getNewServiceClient()->contacts()->get($id);
    }
}

// app/Import/NewService/OrderImporter.php
class OrderImporter extends NewServiceResourcesImporter
{
    public function upsert($data, array $metadata): Model
    {
        $order = NewServiceOrder::firstOrNew([
            'id' => data_get($data, 'id'),
        ]);

        $order->fillTransformedData($data, [
            'amount' => data_get($data, 'amount'),
            'status' => data_get($data, 'status'),
            'new_service_contact_id' => data_get($data, 'contact_id'),
            // ... other transformed fields
        ], $metadata)->save();

        // Import related contact if not exists
        if (!$order->contact && $order->new_service_contact_id) {
            app(ContactImporter::class)->get($order->new_service_contact_id);
        }

        return $order;
    }

    protected function getList(array $filters = []): OffsetPaginator
    {
        return $this->getNewServiceClient()->orders()->paginate($filters);
    }

    protected function getSingleResource(string $id): array
    {
        return $this->getNewServiceClient()->orders()->get($id);
    }
}

Step 5: Create Import Command

// app/Console/Commands/ImportNewServiceResource.php
class ImportNewServiceResource extends BaseCommand
{
    protected $signature = 'new-service:import-resource
                            {resource? : The resource type to import}
                            {--id= : The ID to import}
                            {--updated-minutes-ago= : Filter by recent updates}
                            {--silent : Run in silent mode (no output)}';

    protected $description = 'Import resources from NewService';

    public function handle()
    {
        $importer = $this->argument('resource') ?? (! $this->option('silent') ? select(
            'Select a resource to import',
            collect(NewServiceResourcesImporter::getAvailableImporters())->mapWithKeys(function (string $importer) {
                return [
                    $importer => class_basename($importer),
                ];
            })->all()
        ) : null);

        if (! $importer) {
            $this->error('Resource argument is required when using --silent flag');
            return 1;
        }

        $id = $this->option('id') ?? (! $this->option('silent') ? text(
            'Enter an ID to import specific resource',
            placeholder: 'Press enter to import all resources'
        ) : null);

        $updatedAt = $this->option('updated-minutes-ago');
        $updatedAt = $updatedAt ? now()->subMinutes($updatedAt) : null;

        $importer = app($importer);
        $filters = [];

        if ($updatedAt) {
            $filters['updated_since'] = $updatedAt->toISOString();
        }

        try {
            // Execute the import
            if ($id) {
                $importer->get($id);
            } else {
                $importer->import($filters);
            }

            if (! $this->option('silent')) {
                $this->info('Import completed successfully!');
            }
        } catch (Exception $e) {
            $this->error('Import failed: ' . $e->getMessage());
            return 1;
        }

        return 0;
    }
}

Step 6: Create Migrations

Use the ResourceImporterColumns helper for consistent column structure:

// database/migrations/create_new_service_contacts_table.php

Schema::create('new_service_contacts', function (Blueprint $table) {
    $table->string('id')->primary();
    $table->string('name')->nullable();
    $table->string('email')->nullable();
    $table->string('phone')->nullable();
    $table->timestamps();

    // Add common resource importer columns
    $table->resourceImporter();

    $table->index('email');
});

Step 7: Create Workflows (Optional)

Create workflow classes for orchestrated imports:

// app/Workflows/ImportNewServiceWorkflow.php
#[Dispatchable]
class ImportNewServiceWorkflow extends Workflow
{
    public static $category = 'NewService';

    public function execute(NewServiceResourcesImporter $importer, ?Carbon $updatedAt = null)
    {
        yield ActivityStub::make(ImportNewServiceActivity::class, $importer, $updatedAt);
    }
}

Step 8: Configuration

Service configuration is handled by the service class itself as per the Saloon SDK Implementation Guide. The importer will use the service class to fetch data.

Incremental Importing and Scheduling

Why Incremental Importing is Recommended

Incremental importing (using timestamps to fetch only recently updated data) is strongly recommended for production environments: Benefits:

Reduced API calls: Only fetch changed data since last sync
Lower latency: Faster sync times for regular updates
Better resource utilization: Less CPU, memory, and network usage
Improved reliability: Smaller batches are less likely to timeout or fail
Cost efficiency: Many APIs charge based on request volume

Implementation Pattern:

// Usage in import command
$importer = app(ExternalServiceResourcesImporter::class);
$filters = [];

if ($updatedAt = $this->option('updated-minutes-ago')) {
    $filters['updated_since'] = now()->subMinutes($updatedAt)->toISOString();
}

$importer->import($resource, $filters);

Scheduling Commands

Schedule both incremental and full imports in your routes/console.php. Fortnox is an example.

<?php

use Illuminate\Support\Facades\Schedule;
use App\Console\Commands\ImportFortnoxResource;

// External Service sync - staggered timing to avoid API rate limits
Schedule::command(ImportFortnoxResource::class, ['Order', '--silent', '--updated-minutes-ago' => 120])->withoutOverlapping()->hourlyAt(15);
Schedule::command(ImportFortnoxResource::class, ['Order', '--silent'])->withoutOverlapping()->dailyAt(1);

Schedule::command(ImportFortnoxResource::class, ['Customer', '--silent', '--updated-minutes-ago' => 120])->withoutOverlapping()->hourlyAt(25);
Schedule::command(ImportFortnoxResource::class, ['Customer', '--silent'])->withoutOverlapping()->dailyAt(2);

Scheduling Best Practices

Staggered Timing: Schedule different resource types at different times to avoid API rate limits
Incremental Frequency: Run incremental imports frequently (every hour or few hours)
Full Sync Frequency: Run full imports daily or weekly as a safety net
Overlap Prevention: Always use withoutOverlapping() to prevent concurrent imports
Silent Mode: Use --silent flag for scheduled tasks to reduce log noise

This scheduling approach ensures efficient, reliable, and maintainable data synchronization in production environments.

Triggering Imports by ID

For scenarios where you need to import specific records synchronously (e.g., webhooks, API endpoints, or manual triggers), use the importer’s get() method.

Basic Usage

// In a controller or webhook handler
use App\Import\ExternalService\ExternalServiceResourcesImporter;

public function syncCustomer(string $customerId)
{
    $importer = app(ExternalServiceResourcesImporter::class);

    try {
        $importer->get('Customer', $customerId);
        return response()->json(['message' => 'Customer synced successfully']);
    } catch (Exception $e) {
        traceflow()->error("Failed to sync customer {$customerId}", $e);
        return response()->json(['error' => 'Sync failed'], 500);
    }
}

Structure Overview

The new importer architecture follows a clean three-layer hierarchy:

ResourceImporter (abstract base)
├── importWithPaginator()
├── importSingle()
├── getAvailableImporters()
└── HandlesException trait
    ↓
Service-Specific Base Importers
├── HubSpotResourcesImporter
│   ├── import(filters)
│   ├── get(id)
│   └── abstract methods: upsert(), getList(), getSingleResource()
├── BusinessCentralResourcesImporter
│   ├── import(company)
│   ├── get(company, id)
│   └── abstract methods: upsert(), getAllPaginated(), getById()
└── NewServiceResourcesImporter
    ├── import(filters)
    ├── get(id)
    └── abstract methods: upsert(), getList(), getSingleResource()
        ↓
Individual Resource Importers (concrete implementations)
├── HubSpot/
│   ├── ContactImporter
│   ├── DealImporter
│   └── CompanyImporter
├── BusinessCentral/
│   ├── ItemImporter
│   ├── ColorImporter
│   ├── SizeImporter
│   └── ItemVariantImporter
└── NewService/
    ├── ContactImporter
    ├── OrderImporter
    └── ProductImporter

Key Benefits:

Auto-Discovery: getAvailableImporters() automatically finds all importer classes in a directory
Separation of Concerns: Each resource type has its own dedicated importer class
Service-Specific Logic: Base importers handle service-specific authentication and API patterns
Common Infrastructure: ResourceImporter provides pagination, error handling, and logging
Model Integration: HasResourceObject trait provides ETL methods and display capabilities for resource models
Workflow Support: Importers integrate seamlessly with Laravel workflows for orchestration
Clear Contracts: ResourceObjectContract defines the interface for all resource/integration models

This architecture provides a robust, scalable foundation for integrating with any external service while maintaining clear separation of concerns and comprehensive logging throughout the process.

Architecture

Features

Other

​Importer Implementation Guide

​Overview

​Architecture Components

​1. Extract-Transform-Load (ETL) Pattern

​Extract Phase

​Transform Phase

​Load Phase

​2. Model Structure

​Domain Models (app/Models/)

​External Object Models (app/Models/)

​3. Importer Architecture

​Layer 1: ResourceImporter (Abstract Base)

​Layer 2: Service-Specific Base Importers

​Layer 3: Individual Resource Importers

​4. Service Classes

​5. Command Implementation

​Master Import Command

​Service-Specific Import Commands

​Usage Examples

​6. Workflow Integration

​7. Traceflow Logging System

​8. Implementation Guide for New Services

​Step 1: Create Service Class

​Step 2: Create External Object Models

​Step 3: Create Service-Specific Base Importer

​Step 4: Create Individual Resource Importers

​Step 5: Create Import Command

​Step 6: Create Migrations

​Step 7: Create Workflows (Optional)

​Step 8: Configuration

​Incremental Importing and Scheduling

​Why Incremental Importing is Recommended

​Scheduling Commands

​Scheduling Best Practices

​Triggering Imports by ID

​Basic Usage

​Structure Overview