Skip to main content

Unity Crab Battery

Overview​

The Unity Crab Battery (CRAB) is a comprehensive cognitive assessment tool developed in Unity as part of the Legends of Hoamanu project. CRAB is an interactive application where players help a main character navigate through trials to find a hidden treasure, with each mini-game designed to evaluate specific cognitive skills. The TTF-DDG has significantly contributed to upgrading and improving this system, with major work focused on organization, standardization, reliability, and Android tablet optimization.

Status: ✅ Upgraded & Improved
Platform: Unity (Android tablets)
Type: Cognitive battery assessment (game-based)
Year: 2023 (ongoing development)
Project: CRAB - Legends of Hoamanu

System Description​

Purpose​

The Crab Battery provides:

  • Game-based cognitive assessment: Transforms traditional testing into engaging gameplay
  • Executive function evaluation: Assess key cognitive skills through interactive mini-games
  • Standardized testing protocols: Consistent measurement across participants
  • Cross-platform deployment: Primary focus on Android tablets for field research
  • Reliable data collection: Automated recording of performance metrics
  • Clinical & research applications: Support for speech therapy and cognitive training development

Cognitive Domains​

The battery evaluates executive functions including:

  • Working memory: Temporary storage and manipulation of information
  • Visual span: Visual-spatial memory capacity
  • Visual attention: Ability to focus on and process visual information
  • Other executive functions: Various cognitive control processes

Game-Based Approach​

CRAB uses interactive game mechanics to:

  • Transform assessment into an engaging experience for participants
  • Maintain participant motivation throughout testing
  • Collect cognitive performance data naturally through gameplay
  • Make testing accessible to diverse populations, including children

TTF-DDG Contributions​

The TTF-DDG (particularly Hugo) has invested substantial effort into upgrading and improving the Unity Crab Battery across multiple dimensions. Every commit from Hugo in the crab_unity_project represents a contribution from the TTF-DDG, demonstrating the depth and breadth of this collaborative effort.

Major Improvement Areas​

1. Code Organization & Architecture Refactoring​

Challenge: Original system lacked clear organizational structure, making maintenance difficult
TTF-DDG Solutions:

  • Namespace restructuring: Reorganized code into logical namespaces (Core, GameLogic, SaveSystem, Services, CrabTask, Utils, Network)
  • Assembly definitions: Created modular assembly definition files for better compilation control
  • File organization: Moved files to match namespace structure, one class per file
  • Cognitive complexity reduction: Reduced complexity in key managers (e.g., I_Manager, K_Manager from 110 to 37)
  • Dependency management: Broke circular dependencies, reduced coupling between GameManager and other systems
  • Region-based structuring: Improved code readability with clear section organization

Impact:

  • Easier maintenance and updates (59+ refactoring commits)
  • Faster onboarding for new developers
  • Reduced technical debt significantly
  • Better collaboration capabilities
  • Clearer separation of concerns

2. Testing Infrastructure & Quality Assurance​

Challenge: Limited testing infrastructure, making changes risky
TTF-DDG Solutions:

  • Unit test framework: Implemented comprehensive Play Mode tests for online services
  • Test coverage: Created tests for SaveSystem, ASR components, and core functionality
  • Assembly testing: Added assembly definition files to support proper test isolation
  • Test organization: Moved test files to match script structure for maintainability
  • Quality metrics: Established testing standards for new features

Specific test implementations:

  • SaveSystem unit tests
  • ASR (Automatic Speech Recognition) unit tests
  • Online services Play Mode tests
  • Data validation tests

Impact:

  • Dramatically improved reliability and stability
  • Confidence in making architectural changes
  • Early bug detection in development
  • Documented expected behavior through tests
  • Reduced debugging time significantly
  • Foundation for continuous integration

3. Android Tablet Reliability & Platform Optimization​

Challenge: Inconsistent performance across Android devices
TTF-DDG Solutions:

  • Build optimization: Fixed APK build issues (x64 architecture configuration)
  • Platform-specific fixes: Resolved iOS compiler issues with microphone permissions
  • Unity updates: Managed Unity version updates and compatibility (multiple Unity update commits)
  • Shader updates: Updated Shader Material Pack for compatibility
  • Asset management: Implemented Git LFS for proper binary file tracking
  • CI/CD infrastructure: Set up GitLab CI/CD pipeline (later disabled pending runner setup)

Impact:

  • Consistent experience across Android tablets
  • Reduced crashes and build failures
  • Reliable deployment to devices
  • Professional user experience
  • Better version control for large assets

4. Advanced Speech Recognition (ASR) Integration​

Challenge: Integrate reliable speech recognition for cognitive tasks
TTF-DDG Solutions:

  • Multiple ASR implementations: Evaluated and integrated Whisper, Wav2Vec2, WavLM, and IBM Watson
  • Wav2Vec2 integration: Successfully integrated Wav2Vec2Phoneme with Unity Sentis (neural inference)
  • French language support: Implemented French phonemizer and language models
  • Audio processing: Added audio z-normalization, softmax output processing, volume detection
  • Phoneme alignment: Implemented Needleman-Wunsch algorithm for target phoneme detection
  • ASR scoring system: Separated ASR scoring from task management for modularity
  • Plugin cleanup: Removed deprecated IBM Watson and Whisper plugins when transitioning to Wav2Vec2/WavLM

Key achievements:

  • End-to-end ASR pipeline in Unity
  • Offline speech recognition (no internet required)
  • Multi-language support (French implemented)
  • Real-time transcription capabilities

Impact:

  • Enables speech-based cognitive assessment tasks
  • Offline functionality for field research
  • Standardized ASR interface across tasks
  • Foundation for language-based executive function tests

5. Data Management & Save System Overhaul​

Challenge: Complex, tightly-coupled save system difficult to maintain
TTF-DDG Solutions:

  • Save format modernization: Migrated to .dat.json format for better readability and debugging
  • Data architecture refactoring: Separated trial definition from core logic
  • Upload system abstraction: Created abstract TaskUploader for unified data upload handling
  • Backup system improvements: Enhanced backup reliability, preventing data loss
  • Network handling: Improved internet connectivity checks and data synchronization
  • SQL integration: Organized SQL handlers into dedicated namespace
  • Exception handling: Improved try-catch blocks to preserve stack traces for debugging

Impact:

  • More reliable data collection and storage
  • Easier debugging of data issues
  • Consistent upload mechanism across all tasks
  • Better data recovery capabilities
  • Reduced data loss incidents

6. Documentation & Developer Experience​

Challenge: Limited documentation made collaboration difficult
TTF-DDG Solutions:

  • Code documentation: Added comprehensive inline documentation across codebase
  • Architecture documentation: Created architecture files and CHANGELOG
  • CONTRIBUTING.md: Established contribution guidelines
  • README updates: Maintained up-to-date feature documentation
  • EditorConfig: Added .editorconfig for consistent code formatting

Impact:

  • Easier onboarding for new developers
  • Better collaboration between team members
  • Preserved institutional knowledge
  • Consistent code style across contributors

Technical Architecture​

Unity Implementation​

Project Structure​

After TTF-DDG refactoring, the project follows a clean, modular architecture:

CrabBattery/
├── Core/                      # Core system components and utilities
├── GameLogic/                 # Game logic and GameManager
├── SaveSystem/               # Data persistence and backup
│   ├── Trial/                # Trial data structures
│   ├── Backup/               # Backup system
│   ├── Sql/                  # SQL handlers
│   └── Data/                 # Data models
├── Services/                 # Online services (upload, transcription)
├── Network/                  # HTTP client and networking
├── CrabTask/                 # Individual cognitive test implementations
│   ├── A_RAN/
│   ├── C_Corsi/
│   ├── G_TextReading/
│   ├── I_Manager/
│   ├── K_Vocabulary/
│   └── [Other tasks...]
├── Utils/                    # Shared utilities and helpers
├── ASR/                      # Speech recognition system
└── Tests/                    # Unit and integration tests

Key Components​

  1. Task Framework: Standardized CrabTask base for cognitive tests
  2. SaveSystem: Complete data management with .dat.json format
  3. ASR Pipeline: Wav2Vec2/WavLM integration with Unity Sentis
  4. Services Layer: Modular upload and network services
  5. Assembly Definitions: Proper compilation boundaries (Core, SaveSystem, Services, Utils, ASR, Network)

Technology Stack​

  • Engine: Unity (actively maintained with version updates)
  • Language: C#
  • AI/ML: Unity Sentis (neural inference for ASR)
  • ASR Models: Wav2Vec2Phoneme, WavLM
  • Testing: Unity Test Framework (Play Mode tests)
  • Platform: Android (primary), iOS support considerations
  • Data Format: JSON (.dat.json for saves)
  • Version Control: Git with LFS for large assets
  • CI/CD: GitLab CI/CD (infrastructure prepared)

Quality Assurance​

Testing Infrastructure​

Unit Tests​

  • Component-level testing
  • Logic validation
  • Data handling verification
  • Edge case coverage

Integration Tests​

  • Inter-component communication
  • Task flow validation
  • Data pipeline integrity
  • User interaction scenarios

Platform Testing​

  • Android device matrix testing
  • Performance benchmarking
  • Touch input validation
  • Display compatibility

Quality Metrics​

The TTF-DDG improvements have resulted in measurable quality gains:

  • Code organization: 59+ refactoring commits focused on structure
  • Cognitive complexity: Reduced from 110 to 37 in key managers (66% reduction)
  • Test coverage: Unit tests added for SaveSystem, ASR, and Services
  • Build stability: APK build issues resolved, x64 architecture properly configured
  • Documentation: Comprehensive inline documentation, CHANGELOG, README, CONTRIBUTING.md
  • Modularity: 6+ assembly definition files for clear compilation boundaries
  • Data reliability: New save format with backup system prevents data loss

Deployment & Distribution​

Android Tablet Deployment​

Supported Devices​

[List of tested/supported Android tablets]

System Requirements​

  • Android version: [Minimum version]
  • Screen size: [Requirements]
  • RAM: [Minimum]
  • Storage: [Space needed]

Installation Process​

  1. [Installation steps]
  2. [Configuration]
  3. [Testing/validation]

Build Pipeline​

  • Automated builds
  • Version management
  • Testing before deployment
  • Distribution to research sites

Research Applications​

Assessed Cognitive Domains​

[Specific cognitive domains measured by the battery]

Target Populations​

[Populations for which this battery is designed]

Use Cases​

  • Comprehensive cognitive assessment
  • Longitudinal studies
  • Cross-cultural research
  • Developmental studies
  • Clinical populations

Data Management​

Data Collection​

  • Automated response recording
  • Timing accuracy: millisecond precision
  • Session metadata capture
  • Quality indicators

Data Format​

  • Structured output format
  • Easy analysis integration
  • Anonymization support
  • Export capabilities

Data Quality​

  • Validation during collection
  • Quality flags and indicators
  • Automated quality checks
  • Outlier detection

Reliability Improvements​

Before TTF-DDG Upgrades​

  • Inconsistent architecture: Tightly coupled code, unclear namespace structure
  • Limited testing: No systematic unit testing framework
  • Difficult to maintain: Cognitive complexity >100 in key managers
  • Platform-specific issues: APK build failures, asset management problems
  • Manual quality assurance: No automated testing infrastructure
  • Deprecated dependencies: IBM Watson, outdated video player plugins
  • Data reliability concerns: Complex save system prone to issues

After TTF-DDG Upgrades​

  • ✅ Clean architecture: Modular namespaces (Core, GameLogic, SaveSystem, Services, CrabTask, Utils, ASR, Network)
  • ✅ Comprehensive testing: Unit tests for SaveSystem, ASR, and online services
  • ✅ Maintainable codebase: Cognitive complexity reduced to 37 (66% reduction)
  • ✅ Android-optimized: APK builds work reliably, Git LFS for assets
  • ✅ Automated quality assurance: Test framework with assembly definitions
  • ✅ Modern ASR pipeline: Wav2Vec2/WavLM with Unity Sentis (offline capable)
  • ✅ Robust data handling: .dat.json format with backup system

Measurable Impact​

  • Refactoring commits: 59+ focused on architecture and organization
  • Complexity reduction: 66% decrease in key manager complexity (110→37)
  • Test coverage: Unit tests covering critical systems (SaveSystem, ASR, Services)
  • Build reliability: APK configuration fixed, Git LFS implemented
  • Code quality: 6+ assembly definitions for proper modularity
  • Documentation: CHANGELOG, README, CONTRIBUTING.md, inline docs throughout
  • Feature additions: Complete ASR pipeline with multi-language support

Documentation​

For Researchers​

  • Battery administration manual
  • Scoring and interpretation guide
  • Data analysis guide
  • Troubleshooting tips
  • Device setup instructions

For Developers​

  • Technical architecture documentation
  • Development guidelines
  • Testing procedures
  • Build and deployment guide
  • Contributing guidelines

Access documentation →

Availability​

Access​

The Unity Crab Battery is available through:

Support​

The TTF-DDG provides:

  • Technical support
  • Training for administrators
  • Troubleshooting assistance
  • Custom adaptations (if needed)

Contact us for support →

Future Development​

Planned Enhancements​

  • Additional cognitive tasks
  • Enhanced analytics capabilities
  • iOS version development
  • Cloud synchronization
  • Multi-language support

Ongoing Maintenance​

  • Regular Unity version updates
  • Android platform compatibility
  • Performance optimization
  • Bug fixes and improvements
  • Security updates

Lessons Learned​

Technical Insights​

The Unity Crab Battery upgrades demonstrate several critical principles:

  • Refactoring is Essential: Reducing cognitive complexity from 110→37 dramatically improved maintainability
  • Testing Enables Confidence: Unit tests allowed safe architectural changes without breaking functionality
  • Modularity Through Assembly Definitions: Proper compilation boundaries prevent unwanted dependencies
  • Namespace Organization: Clear structure (Core, GameLogic, SaveSystem, etc.) makes navigation intuitive
  • Documentation as Code: Inline docs, CHANGELOG, and architecture files preserve institutional knowledge
  • Data Format Matters: .dat.json format is debuggable and human-readable compared to binary formats
  • Offline-First ASR: Wav2Vec2/WavLM with Unity Sentis enables field research without internet dependency
  • Version Control for Assets: Git LFS is crucial for Unity projects with large binary assets

Best Practices Established​

  1. Unity Architecture Standards:

    • One class per file
    • Namespace-to-folder mapping
    • Assembly definitions for major subsystems
    • Region-based code organization

  2. Refactoring Methodology:

    • Measure cognitive complexity before/after
    • Break circular dependencies systematically
    • Extract interfaces for testability
    • Preserve stack traces in exception handling

  3. Testing Strategy:

    • Play Mode tests for Unity-dependent components
    • Assembly-level test isolation
    • Test core systems (SaveSystem, ASR, Services) first

  4. Mobile Development:

    • Platform-specific compilation directives
    • x64 architecture for neural inference
    • Git LFS for shader and asset management

  5. ASR Integration:

    • Separate ASR logic from task logic
    • Phoneme-level processing for language tasks
    • Offline model deployment with Unity Sentis

These practices now inform all TTF-DDG Unity development and serve as a model for upgrading legacy research software.

Impact​

For Researcher Users​

  • Reliable cognitive assessment tool
  • Consistent administration across sites
  • High-quality data collection
  • Portable, tablet-based deployment

For NCCR​

  • Professional-grade assessment battery
  • Cross-work package utility
  • Demonstrated quality improvement
  • Model for other Unity projects

For the Field​

  • Example of research software quality
  • Open methodologies (if applicable)
  • Replicable protocols
  • Technical documentation

The Crab Battery work connects with:

  • ASR Collection - Speech-to-text also mentioned in context of this work
  • Library of Tasks - Available through the library
  • Other Unity-based tasks in development

Acknowledgments​

Original Development: Zeno Mestrina in partnership with University of Geneva
Upgrades & Improvements: TTF-DDG (Hugo - primary contributor)
Project Context: CRAB (Cognitive Research Assessment Battery) - Legends of Hoamanu
Collaboration: NCCR Evolving Language & University of Geneva

The extensive refactoring, testing infrastructure, ASR integration, and quality improvements represent a significant investment by the TTF-DDG in transforming CRAB from a prototype into a robust, maintainable research tool.

← Back to Tasks | View other implementations →