neo/docs/SYSTEM_FIELDS_FIX.md

# System Fields Validation Fix - Checklist

## Problem
When creating or updating records, frontend validation was showing:
- "Created At is required"
- "Updated At is required"

This happened because system-managed fields were marked with `isRequired: true` in the database and frontend was trying to validate them.

## Root Causes Identified

1. **Backend Issue**: Standard field definitions were created with `isRequired: true`
   - `ownerId` - marked required but auto-set by system
   - `created_at` - marked required but auto-set by system  
   - `updated_at` - marked required but auto-set by system
   - `name` - marked required but should be optional

2. **Backend Issue**: System fields not marked with `isSystem: true`
   - Missing flag that identifies auto-managed fields
   - Frontend couldn't distinguish system fields from user fields

3. **Frontend Issue**: Field hiding logic didn't fully account for system fields
   - Only checked against hardcoded list of field names
   - Didn't check `isSystem` flag from backend

4. **Frontend Issue**: Form data wasn't filtered before saving
   - System fields might be included in submission
   - Could cause validation errors on backend

## Fixes Applied

### Backend Changes

**File**: [backend/src/object/object.service.ts](backend/src/object/object.service.ts#L100-L142)

Changed standard field definitions:
```typescript
// BEFORE (lines 100-132)
ownerId:      isRequired: true
name:         isRequired: true
created_at:   isRequired: true
updated_at:   isRequired: true

// AFTER 
ownerId:      isRequired: false, isSystem: true
name:         isRequired: false, isSystem: false
created_at:   isRequired: false, isSystem: true
updated_at:   isRequired: false, isSystem: true
```

Changes made:
- ✅ Set `isRequired: false` for all system fields (they're auto-managed)
- ✅ Added `isSystem: true` flag for ownerId, created_at, updated_at
- ✅ Set `isCustom: false` for all standard fields
- ✅ Set `name` as optional field (`isRequired: false`)

### Frontend Changes

**File**: [frontend/composables/useFieldViews.ts](frontend/composables/useFieldViews.ts#L12-L40)

Enhanced field mapping logic:
```typescript
// BEFORE
const isAutoGeneratedField = ['id', 'createdAt', 'updatedAt', 'createdBy', 'updatedBy']

// AFTER
const isSystemField = Boolean(fieldDef.isSystem)  // Check backend flag
const isAutoGeneratedField = ['id', 'createdAt', 'updatedAt', 'created_at', 'updated_at', 'createdBy', 'updatedBy']
const shouldHideOnEdit = isSystemField || isAutoGeneratedField  // Check both

showOnEdit: fieldDef.uiMetadata?.showOnEdit ?? !shouldHideOnEdit  // Hide system fields
```

Changes made:
- ✅ Added check for backend `isSystem` flag
- ✅ Added snake_case field names (created_at, updated_at)
- ✅ Combined both checks to hide system fields on edit
- ✅ System fields still visible on list and detail views (read-only)

**File**: [frontend/components/views/EditViewEnhanced.vue](frontend/components/views/EditViewEnhanced.vue#L160-L169)

Added data filtering before save:
```typescript
// BEFORE
const handleSave = () => {
  if (validateForm()) {
    emit('save', formData.value)
  }
}

// AFTER
const handleSave = () => {
  if (validateForm()) {
    // Filter out system fields from save data
    const saveData = { ...formData.value }
    const systemFields = ['id', 'tenantId', 'ownerId', 'created_at', 'updated_at', 'createdAt', 'updatedAt']
    for (const field of systemFields) {
      delete saveData[field]
    }
    emit('save', saveData)
  }
}
```

Changes made:
- ✅ Strip system fields before sending to API
- ✅ Prevents accidental submission of read-only fields
- ✅ Ensures API receives only user-provided data

## How It Works Now

### Create Record Flow
```
User fills form with business data:
  { name: "Acme", revenue: 1000000 }
         ↓
Frontend validation skips system fields:
  - created_at (showOnEdit: false, filtered)
  - updated_at (showOnEdit: false, filtered)
  - ownerId (showOnEdit: false, filtered)
         ↓
Frontend filters system fields before save:
  deleteProperty(saveData, 'created_at')
  deleteProperty(saveData, 'updated_at')
  deleteProperty(saveData, 'ownerId')
         ↓
API receives clean data:
  { name: "Acme", revenue: 1000000 }
         ↓
Backend's Objection model auto-manages:
  $beforeInsert() hook:
    - Sets id (UUID)
    - Sets ownerId (from userId)
    - Sets created_at (now)
    - Sets updated_at (now)
         ↓
Database receives complete record with all fields
```

### Update Record Flow
```
User edits record, changes revenue:
  { revenue: 1500000 }
         ↓
Frontend validation skips system fields
Frontend filters before save:
  - Removes ownerId (read-only)
  - Removes created_at (immutable)
  - Removes updated_at (will be set by system)
         ↓
API receives:
  { revenue: 1500000 }
         ↓
Backend filters out protected fields (double-check):
  delete allowedData.ownerId
  delete allowedData.created_at
  delete allowedData.tenantId
         ↓
Backend's Objection model:
  $beforeUpdate() hook:
    - Sets updated_at (now)
         ↓
Database receives update with timestamp updated
```

## Field Visibility Rules

System fields now properly hidden:

| Field | Create | Detail | List | Edit | Notes |
|-------|--------|--------|------|------|-------|
| id | No | Yes | No | No | Auto-generated UUID |
| ownerId | No | Yes | No | No | Auto-set from auth |
| created_at | No | Yes | Yes | No | Auto-set on insert |
| updated_at | No | Yes | No | No | Auto-set on insert/update |
| name | No | Yes | Yes | **Yes** | Optional user field |
| custom fields | No | Yes | Yes | Yes | User-defined fields |

Legend:
- No = Field not visible to users
- Yes = Field visible (read-only or editable)

## Backend System Field Management

Standard fields auto-created for every new object:

```
ownerId (type: LOOKUP)
  ├─ isRequired: false
  ├─ isSystem: true
  ├─ isCustom: false
  └─ Auto-set by ObjectService.createRecord()

name (type: TEXT)
  ├─ isRequired: false
  ├─ isSystem: false
  ├─ isCustom: false
  └─ Optional user field

created_at (type: DATE_TIME)
  ├─ isRequired: false
  ├─ isSystem: true
  ├─ isCustom: false
  └─ Auto-set by DynamicModel.$beforeInsert()

updated_at (type: DATE_TIME)
  ├─ isRequired: false
  ├─ isSystem: true
  ├─ isCustom: false
  └─ Auto-set by DynamicModel.$beforeInsert/Update()
```

## Validation Logic

### Frontend Validation (EditViewEnhanced.vue)

1. Skip fields with `showOnEdit === false`
   - System fields automatically excluded
   - Created At, Updated At, ownerId won't be validated

2. Validate only remaining fields:
   - Check required fields have values
   - Apply custom validation rules
   - Show errors inline

3. Filter data before save:
   - Remove system fields
   - Send clean data to API

### Backend Validation (ObjectService)

1. Check object definition exists
2. Get bound Objection model
3. Model validates field types (JSON schema)
4. Model auto-manages system fields via hooks
5. Insert/Update data in database

## Testing the Fix

### Test 1: Create Record
```bash
# In Nuxt app, create new record
POST /api/records/Account
Body: { 
  name: "Test Account",
  revenue: 1000000
}

# Should NOT show validation error for Created At or Updated At
# Should create record with auto-populated system fields
```

### Test 2: Check System Fields Are Hidden
```
Look at create form:
- ✅ ownerId field - NOT visible
- ✅ created_at field - NOT visible
- ✅ updated_at field - NOT visible
- ✅ name field - VISIBLE (optional)
- ✅ custom fields - VISIBLE
```

### Test 3: Update Record
```bash
# Edit existing record
PATCH /api/records/Account/record-id
Body: {
  revenue: 1500000
}

# Should NOT show validation error
# Should NOT allow changing ownerId
# Should auto-update timestamp
```

### Test 4: Verify Frontend Filtering
```
Open browser console:
- Check form data before save
- Should NOT include id, ownerId, created_at, updated_at
- Should include user-provided fields only
```

## Files Modified

| File | Changes | Status |
|------|---------|--------|
| [backend/src/object/object.service.ts](backend/src/object/object.service.ts) | Standard fields: isRequired→false, added isSystem, isCustom | ✅ |
| [frontend/composables/useFieldViews.ts](frontend/composables/useFieldViews.ts) | Field hiding logic: check isSystem flag + snake_case names | ✅ |
| [frontend/components/views/EditViewEnhanced.vue](frontend/components/views/EditViewEnhanced.vue) | handleSave: filter system fields before emit | ✅ |

## Verification

✅ Backend compiles: `npm run build` successful
✅ System fields marked with isSystem: true
✅ System fields marked with isRequired: false
✅ Frontend filtering implemented
✅ Frontend hiding logic enhanced

## Related Documentation

- [OBJECTION_MODEL_SYSTEM.md](OBJECTION_MODEL_SYSTEM.md) - Model system details
- [OBJECTION_QUICK_REFERENCE.md](OBJECTION_QUICK_REFERENCE.md) - Quick guide
- [TEST_OBJECT_CREATION.md](TEST_OBJECT_CREATION.md) - Test procedures

## Summary

The fix ensures that system-managed fields (id, ownerId, created_at, updated_at) are:
1. **Never required from users** - Marked `isRequired: false`
2. **Clearly marked as system** - Have `isSystem: true` flag
3. **Hidden from edit forms** - Via `showOnEdit: false`
4. **Filtered before submission** - Not sent to API
5. **Auto-managed by backend** - Set by model hooks
6. **Protected from modification** - Backend filters out in updates