CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

SensorFabric UltraHuman (UH) is an AWS Lambda-based data pipeline that collects health/fitness metrics from UltraHuman devices. The system consists of four main Lambda functions:

biobayb_uh_publisher: Fetches active participants from MDH and publishes SNS messages to trigger data collection (scheduled daily)
biobayb_uh_uploader: Processes SNS messages, fetches UltraHuman data via API, and uploads parquet files to S3
biobayb_uh_template_generator: Generates weekly health reports from collected data with JWT authentication
biobayb_uh_jwt_generator: Generates JWT tokens for secure report access and updates MDH participant custom fields

Build and Deployment Commands

Primary Deployment

# Full build and deploy pipeline (recommended)
./deploy.sh

# Build and deploy using CDK
./deploy.sh --cdk

# Deploy to specific stack only
./deploy.sh --stack uh-biobayb-dev

# Build container images only (no deployment)
./deploy.sh --build-only

# Deploy only (assumes images exist in ECR)
./deploy.sh --deploy-only

Testing and Validation

# Run health check on deployed functions
./deploy.sh --health-check

# Test deployed functions
./deploy.sh --test

# Rollback to previous deployment
./deploy.sh --rollback

CDK Operations

cd cdk/
pip install -r requirements.txt

# Bootstrap CDK (first time only)
cdk bootstrap

# Deploy infrastructure
cdk deploy --all

# View changes before deployment
cdk diff

# Destroy infrastructure
cdk destroy

Architecture Overview

Data Flow

Scheduled Trigger: EventBridge rule triggers
```
biobayb_uh_publisher
```
daily at 7:00 AM UTC
Participant Discovery: Publisher queries MDH database for active participants
SNS Publishing: Publisher sends SNS message for each participant to
```
mdh_uh_sync
```
topic
Data Collection: SNS message triggers
```
biobayb_uh_uploader
```
Lambda
API Integration: Uploader fetches data from UltraHuman API using participant email
Data Storage: Raw data flattened and stored as parquet files in S3 bucket
MDH Updates: Participant sync timestamps updated in MDH database
Weekly JWT Generation: EventBridge rule triggers
```
biobayb_uh_jwt_generator
```
weekly on Sundays at midnight UTC-7

Key Components

ECR Repository:
```
uh-biobayb
```
- shared Docker image for all Lambda functions
SNS Topic: Inter-function communication via
```
mdh_uh_sync
```
topic
S3 Bucket: Data storage in
```
uoa-biobayb-uh-{env}
```
buckets
Secrets Manager: UltraHuman API credentials and JWT secrets stored in
```
prod/biobayb/uh/keys
```
SQS DLQ: Dead letter queue for failed message processing
CloudWatch: Centralized logging with infinite retention

Environment Configuration

The system supports multiple deployment environments:

Development:
```
uh-biobayb-dev
```
stack using staging UltraHuman API
Production:
```
uh-biobayb-prod
```
stack using production UltraHuman API

Each environment has isolated resources (databases, S3 buckets, SNS topics).

JWT Authentication System

The template generator now uses JWT-based authentication for secure report access:

JWT tokens contain participant_id, start_date, end_date with HS256 signing
Tokens are generated by the JWT generator Lambda and stored in MDH participant custom fields under 'report_jwt'
Template generator validates tokens before generating reports, returning 401 for invalid tokens
REPORT_SECRET in Secrets Manager used for JWT signing and validation

Lambda Function Details

All functions use shared Docker image from ECR with different CMD handlers:

Memory: 2048-3072 MB depending on function
Timeout: 10-15 minutes
Architecture: x86_64
Runtime: Container-based Python

Data Pipeline Structure

S3 data organized by participant and date:

s3://bucket/
├── participant_id=xxx/
│   ├── date=2024-01-01/
│   │   └── data.parquet
│   └── date=2024-01-02/
│       └── data.parquet

Development Notes

Prerequisites

AWS CLI configured with proper credentials
Docker daemon running
CDK CLI installed (
```
npm install -g aws-cdk
```
)
Python 3.9+ with required packages from
```
requirements.txt
```

Environment Variables

Key environment variables used by Lambda functions:

```
UH_ENVIRONMENT
```
: "development" or "production"
```
UH_SNS_TOPIC_ARN
```
: SNS topic ARN for publishing messages
```
SF_DATA_BUCKET
```
: S3 bucket for data storage
```
AWS_SECRET_NAME
```
: Secrets Manager secret name
```
UH_DLQ_URL
```
: Dead letter queue URL

Manual Triggers

EventBridge patterns for manual execution:

Uploader:

{
  "source": ["sensorfabric.manual"],
  "detail-type": ["UltraHuman Data Upload Request"]
}

JWT Generator:

{
  "source": ["sensorfabric.manual"],
  "detail-type": ["UltraHuman JWT Generation Request"],
  "detail": {
    "start_date": "2024-01-01",
    "end_date": "2024-01-31",
    "participant_id": "optional_specific_participant"
  }
}

Monitoring

CloudWatch logs:
```
/aws/lambda/{function_name}
```
Health checks via
```
deploy.sh --health-check
```
Failed executions sent to SQS DLQ with 2 retry attempts

Debugging

# View function logs
aws logs tail /aws/lambda/{function_name} --follow

# Test function locally
aws lambda invoke --function-name {function_name} /tmp/response.json

# Check CloudFormation events
aws cloudformation describe-stack-events --stack-name {stack_name}

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

SensorFabric UltraHuman (UH) is an AWS Lambda-based data pipeline that collects health/fitness metrics from UltraHuman devices. The system consists of four main Lambda functions:

biobayb_uh_publisher: Fetches active participants from MDH and publishes SNS messages to trigger data collection (scheduled daily)
biobayb_uh_uploader: Processes SNS messages, fetches UltraHuman data via API, and uploads parquet files to S3
biobayb_uh_template_generator: Generates weekly health reports from collected data with JWT authentication
biobayb_uh_jwt_generator: Generates JWT tokens for secure report access and updates MDH participant custom fields

Build and Deployment Commands

Primary Deployment

# Full build and deploy pipeline (recommended)
./deploy.sh

# Build and deploy using CDK
./deploy.sh --cdk

# Deploy to specific stack only
./deploy.sh --stack uh-biobayb-dev

# Build container images only (no deployment)
./deploy.sh --build-only

# Deploy only (assumes images exist in ECR)
./deploy.sh --deploy-only

Testing and Validation

# Run health check on deployed functions
./deploy.sh --health-check

# Test deployed functions
./deploy.sh --test

# Rollback to previous deployment
./deploy.sh --rollback

CDK Operations

cd cdk/
pip install -r requirements.txt

# Bootstrap CDK (first time only)
cdk bootstrap

# Deploy infrastructure
cdk deploy --all

# View changes before deployment
cdk diff

# Destroy infrastructure
cdk destroy

Architecture Overview

Data Flow

Scheduled Trigger: EventBridge rule triggers
```
biobayb_uh_publisher
```
daily at 7:00 AM UTC
Participant Discovery: Publisher queries MDH database for active participants
SNS Publishing: Publisher sends SNS message for each participant to
```
mdh_uh_sync
```
topic
Data Collection: SNS message triggers
```
biobayb_uh_uploader
```
Lambda
API Integration: Uploader fetches data from UltraHuman API using participant email
Data Storage: Raw data flattened and stored as parquet files in S3 bucket
MDH Updates: Participant sync timestamps updated in MDH database
Weekly JWT Generation: EventBridge rule triggers
```
biobayb_uh_jwt_generator
```
weekly on Sundays at midnight UTC-7

Key Components

ECR Repository:
```
uh-biobayb
```
- shared Docker image for all Lambda functions
SNS Topic: Inter-function communication via
```
mdh_uh_sync
```
topic
S3 Bucket: Data storage in
```
uoa-biobayb-uh-{env}
```
buckets
Secrets Manager: UltraHuman API credentials and JWT secrets stored in
```
prod/biobayb/uh/keys
```
SQS DLQ: Dead letter queue for failed message processing
CloudWatch: Centralized logging with infinite retention

Environment Configuration

The system supports multiple deployment environments:

Development:
```
uh-biobayb-dev
```
stack using staging UltraHuman API
Production:
```
uh-biobayb-prod
```
stack using production UltraHuman API

Each environment has isolated resources (databases, S3 buckets, SNS topics).

JWT Authentication System

The template generator now uses JWT-based authentication for secure report access:

JWT tokens contain participant_id, start_date, end_date with HS256 signing
Tokens are generated by the JWT generator Lambda and stored in MDH participant custom fields under 'report_jwt'
Template generator validates tokens before generating reports, returning 401 for invalid tokens
REPORT_SECRET in Secrets Manager used for JWT signing and validation

Lambda Function Details

All functions use shared Docker image from ECR with different CMD handlers:

Memory: 2048-3072 MB depending on function
Timeout: 10-15 minutes
Architecture: x86_64
Runtime: Container-based Python

Data Pipeline Structure

S3 data organized by participant and date:

s3://bucket/
├── participant_id=xxx/
│   ├── date=2024-01-01/
│   │   └── data.parquet
│   └── date=2024-01-02/
│       └── data.parquet

Development Notes

Prerequisites

AWS CLI configured with proper credentials
Docker daemon running
CDK CLI installed (
```
npm install -g aws-cdk
```
)
Python 3.9+ with required packages from
```
requirements.txt
```

Environment Variables

Key environment variables used by Lambda functions:

```
UH_ENVIRONMENT
```
: "development" or "production"
```
UH_SNS_TOPIC_ARN
```
: SNS topic ARN for publishing messages
```
SF_DATA_BUCKET
```
: S3 bucket for data storage
```
AWS_SECRET_NAME
```
: Secrets Manager secret name
```
UH_DLQ_URL
```
: Dead letter queue URL

Manual Triggers

EventBridge patterns for manual execution:

Uploader:

{
  "source": ["sensorfabric.manual"],
  "detail-type": ["UltraHuman Data Upload Request"]
}

JWT Generator:

{
  "source": ["sensorfabric.manual"],
  "detail-type": ["UltraHuman JWT Generation Request"],
  "detail": {
    "start_date": "2024-01-01",
    "end_date": "2024-01-31",
    "participant_id": "optional_specific_participant"
  }
}

Monitoring

CloudWatch logs:
```
/aws/lambda/{function_name}
```
Health checks via
```
deploy.sh --health-check
```
Failed executions sent to SQS DLQ with 2 retry attempts

Debugging

# View function logs
aws logs tail /aws/lambda/{function_name} --follow

# Test function locally
aws lambda invoke --function-name {function_name} /tmp/response.json

# Check CloudFormation events
aws cloudformation describe-stack-events --stack-name {stack_name}

CLAUDE.md

Related Skills

Markdown Converter

Nano Banana Pro

1password