Skip to content

python-grpc-phase4-plan.md

Latest commit

 

History

History
1392 lines (1056 loc) · 39 KB

python-grpc-phase4-plan.md

File metadata and controls

1392 lines (1056 loc) · 39 KB
Raw

Phase 4: Runtime APIs Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Expose OpenSysML runtime capabilities (eval, instantiate, execute actions/state machines) via Python client

Architecture: Extend protobuf schema with 4 new RPCs (Evaluate, Instantiate, ExecuteAction, ExecuteState), implement gRPC service handlers calling existing internal/exec/runtime functions, add Python wrapper methods in Connection class plus new Instance class

Tech Stack: Go 1.23+, protobuf 7.35.1+, grpcio 1.83.0+, existing OpenSysML runtime (eval.go, instance.go, action_executor.go, state_executor.go)

File Structure

Go (gRPC Service):

  • api/proto/sysml.proto - Add EvaluateRequest/Response, InstantiateRequest/Response, ExecuteActionRequest/Response, ExecuteStateRequest/Response messages + 4 RPCs
  • internal/frontend/grpc/service.go - Add 4 RPC handlers calling runtime package
  • internal/frontend/grpc/convert.go - Add Value → protobuf conversion (ValueToProto), Instance → protobuf conversion (InstanceToProto)

Python (Client):

  • opensysml/instance.py - New Instance class wrapping protobuf Instance message
  • opensysml/connection.py - Add eval(), instantiate(), execute_action(), execute_state() methods
  • opensysml/errors.py - New RuntimeError exception for execution failures
  • opensysml/__init__.py - Export Instance, RuntimeError, add module-level eval()/instantiate() helpers

Tests:

  • internal/frontend/grpc/runtime_test.go - Go unit tests for 4 new RPCs
  • tests/test_runtime.py - Python unit tests with mocked RPCs
  • tests/test_runtime_integration.py - Integration tests against real service with calc/action/state fixtures

Task Breakdown

Task 1: Extend Protobuf Schema with Runtime RPCs (Go)

Objective: Add 4 new RPC definitions + message types to api/proto/sysml.proto

Files:

  • Modify: api/proto/sysml.proto
  • Generate: api/proto/sysml_pb2.py, api/proto/sysml_pb2_grpc.py (Python stubs)
  • Generate: api/proto/sysml.pb.go, api/proto/sysml_grpc.pb.go (Go stubs)

Task 2: Implement Go Runtime RPC Handlers

Objective: Add Evaluate, Instantiate, ExecuteAction, ExecuteState handlers in internal/frontend/grpc/service.go

Files:

  • Modify: internal/frontend/grpc/service.go
  • Modify: internal/frontend/grpc/convert.go (add ValueToProto, InstanceToProto)
  • Create: internal/frontend/grpc/runtime_test.go (unit tests for 4 RPCs)

Task 3: Implement Python Instance Class

Objective: Create opensysml/instance.py wrapping protobuf Instance message

Files:

  • Create: opensysml/instance.py
  • Create: tests/test_instance.py (unit tests)

Task 4: Add Python Runtime Methods to Connection

Objective: Add eval(), instantiate(), execute_action(), execute_state() methods to Connection class

Files:

  • Create: opensysml/errors.py (RuntimeError exception)
  • Modify: opensysml/connection.py (add 4 runtime methods)
  • Create: tests/test_runtime.py (unit tests with mocked RPCs)

Task 5: Add Module-Level Runtime Helpers

Objective: Add opensysml.eval(), opensysml.instantiate() convenience functions

Files:

  • Modify: opensysml/__init__.py (add eval/instantiate, export Instance/RuntimeError)
  • Modify: tests/test_api.py (add tests for module-level helpers)

Task 6: Integration Testing

Objective: End-to-end tests with real service using calc/action/state fixtures

Files:

  • Create: tests/test_runtime_integration.py (7 integration tests)
  • Verify: All tests pass against real sysml-grpc service

Definition of Done

  • opensysml.eval("2 + 2") returns 4
  • instance = opensysml.instantiate("PartName") returns Instance object
  • result = conn.execute_action("ActionName", inputs={}) returns outputs dict
  • Runtime errors raise opensysml.RuntimeError with message + diagnostics
  • Integration tests pass for all 4 runtime operations
  • All existing tests still pass (no regressions)
  • Go tests pass: go test ./...
  • Python tests pass: pytest tests/

Task 1: Extend Protobuf Schema with Runtime RPCs

Objective: Add 4 new RPC definitions (Evaluate, Instantiate, ExecuteAction, ExecuteState) and their message types to api/proto/sysml.proto

Files:

  • Modify: api/proto/sysml.proto

Step 1: Add Value message to protobuf schema

The runtime Value type (value.go) supports multiple kinds. Map to protobuf:

// Value represents a runtime-evaluable value
message Value {
  oneof kind {
    int64 int_value = 1;
    double real_value = 2;
    bool bool_value = 3;
    string string_value = 4;
    int64 instance_id = 5;      // reference to Instance
    ValueSequence sequence = 6;
    string null = 7;            // marker for null (empty string)
  }
}

message ValueSequence {
  repeated Value elements = 1;
}

Add after message Diagnostic (around line 80 in sysml.proto).

Step 2: Add Evaluate RPC messages

message EvaluateRequest {
  string model_hash = 1;        // from ParseFile response
  string expression = 2;         // SysML expression string (e.g., "2 + 2")
  string context_symbol_id = 3; // optional: symbol FQN for context scope
}

message EvaluateResponse {
  Value result = 1;
  string error = 2;              // non-empty if evaluation failed
  repeated Diagnostic diagnostics = 3;
}

Add after DiagnosticsResponse (around line 90).

Step 3: Add Instance-related messages

message Instance {
  int64 id = 1;
  string type_symbol_id = 2;     // FQN of the def/usage
  map<string, SlotValue> slots = 3;
}

message SlotValue {
  string feature_name = 1;
  Value value = 2;               // for scalar slots
  repeated Value values = 3;     // for collection slots
  bool materialized = 4;
}

message InstantiateRequest {
  string model_hash = 1;
  string symbol_id = 2;          // FQN of part/usage to instantiate
}

message InstantiateResponse {
  Instance instance = 1;
  string error = 2;
  repeated Diagnostic diagnostics = 3;
}

Add after EvaluateResponse.

Step 4: Add ExecuteAction messages

message ExecuteActionRequest {
  string model_hash = 1;
  string action_symbol_id = 2;   // FQN of action def
  map<string, Value> inputs = 3; // parameter name → value
}

message ExecuteActionResponse {
  map<string, Value> outputs = 1; // output parameter name → value
  string error = 2;
  repeated Diagnostic diagnostics = 3;
}

Add after InstantiateResponse.

Step 5: Add ExecuteState messages

message ExecuteStateRequest {
  string model_hash = 1;
  string state_machine_symbol_id = 2;
  repeated string events = 3;    // sequence of event names to process
}

message ExecuteStateResponse {
  repeated string states_visited = 1; // trace of state names
  map<string, Value> final_context = 2;
  string error = 3;
  repeated Diagnostic diagnostics = 4;
}

Add after ExecuteActionResponse.

Step 6: Add 4 new RPCs to service definition

Update the service SysMLService block:

service SysMLService {
  rpc ParseFile(ParseFileRequest) returns (ParseFileResponse);
  rpc GetSymbol(GetSymbolRequest) returns (SymbolResponse);
  rpc GetDiagnostics(DiagnosticsRequest) returns (DiagnosticsResponse);
  
  // Runtime operations (Phase 4)
  rpc Evaluate(EvaluateRequest) returns (EvaluateResponse);
  rpc Instantiate(InstantiateRequest) returns (InstantiateResponse);
  rpc ExecuteAction(ExecuteActionRequest) returns (ExecuteActionResponse);
  rpc ExecuteState(ExecuteStateRequest) returns (ExecuteStateResponse);
}

Step 7: Regenerate protobuf stubs

cd api/proto
go generate

This runs protoc to generate:

  • sysml.pb.go (Go messages)
  • sysml_grpc.pb.go (Go service)

Step 8: Regenerate Python stubs

cd /home/han/IdeaProjects/OpenSysML
python -m grpc_tools.protoc -I api/proto \
  --python_out=opensysml/proto \
  --grpc_python_out=opensysml/proto \
  api/proto/sysml.proto

Generates:

  • opensysml/proto/sysml_pb2.py
  • opensysml/proto/sysml_pb2_grpc.py

Step 9: Fix Python import paths

Edit opensysml/proto/sysml_pb2_grpc.py line 6:

# Change: import sysml_pb2 as sysml__pb2
# To:     from . import sysml_pb2 as sysml__pb2

Step 10: Verify compilation

# Go
go build ./api/proto
go build ./internal/frontend/grpc

# Python
PYTHONPATH=/home/han/IdeaProjects/OpenSysML python -c "from opensysml.proto import sysml_pb2, sysml_pb2_grpc; print('OK')"

Expected: All compile cleanly.

Step 11: Commit

git add api/proto/sysml.proto api/proto/*.go opensysml/proto/*.py
git commit -m "feat(proto): add runtime RPC messages (Evaluate, Instantiate, ExecuteAction, ExecuteState)"

Task 2: Implement Go Runtime RPC Handlers

Objective: Add Evaluate, Instantiate, ExecuteAction, ExecuteState RPC handlers to internal/frontend/grpc/service.go, plus conversion helpers

Files:

  • Modify: internal/frontend/grpc/service.go
  • Modify: internal/frontend/grpc/convert.go
  • Create: internal/frontend/grpc/runtime_test.go

Phase A: Add Conversion Helpers

Step 1: Add ValueToProto conversion function to convert.go

// ValueToProto converts runtime.Value to protobuf Value message
func ValueToProto(val runtime.Value) *pb.Value {
	switch val.Kind {
	case runtime.ValConst:
		// semantics.Value can be int, real, bool, infinity
		if val.Const.IsInt() {
			return &pb.Value{Kind: &pb.Value_IntValue{IntValue: val.Const.IntVal}}
		}
		if val.Const.IsReal() {
			return &pb.Value{Kind: &pb.Value_RealValue{RealValue: val.Const.RealVal}}
		}
		if val.Const.IsBool() {
			return &pb.Value{Kind: &pb.Value_BoolValue{BoolValue: val.Const.BoolVal}}
		}
		// Infinity → use real with special encoding (or string)
		return &pb.Value{Kind: &pb.Value_StringValue{StringValue: "*"}}
		
	case runtime.ValString:
		return &pb.Value{Kind: &pb.Value_StringValue{StringValue: val.Str}}
		
	case runtime.ValNull:
		return &pb.Value{Kind: &pb.Value_Null{Null: ""}}
		
	case runtime.ValInstance:
		return &pb.Value{Kind: &pb.Value_InstanceId{InstanceId: val.Instance}}
		
	case runtime.ValSequence:
		elements := make([]*pb.Value, 0)
		for i := 0; i < val.Sequence.Len(); i++ {
			elem := val.Sequence.At(i)
			elements = append(elements, ValueToProto(elem))
		}
		return &pb.Value{Kind: &pb.Value_Sequence{Sequence: &pb.ValueSequence{Elements: elements}}}
		
	default:
		return &pb.Value{Kind: &pb.Value_Null{Null: "unsupported"}}
	}
}

Add after ParserDiagnosticToProto (around line 80 in convert.go).

Step 2: Add InstanceToProto conversion function

// InstanceToProto converts runtime.Instance to protobuf Instance message
func InstanceToProto(inst *runtime.Instance) *pb.Instance {
	slots := make(map[string]*pb.SlotValue)
	
	for name, slot := range inst.Slots {
		pbSlot := &pb.SlotValue{
			FeatureName:  name,
			Materialized: slot.Materialized,
		}
		
		// Check if scalar or collection based on Feature multiplicity
		if slot.Feature.Multiplicity.Upper.Value <= 1 {
			pbSlot.Value = ValueToProto(slot.Value)
		} else {
			// Collection slot - convert Values field
			if slot.Values.Kind == runtime.ValSequence {
				pbValues := make([]*pb.Value, 0)
				for i := 0; i < slot.Values.Sequence.Len(); i++ {
					pbValues = append(pbValues, ValueToProto(slot.Values.Sequence.At(i)))
				}
				pbSlot.Values = pbValues
			}
		}
		
		slots[name] = pbSlot
	}
	
	return &pb.Instance{
		Id:            inst.ID,
		TypeSymbolId:  inst.Type.Name, // or use idx.GetFQN(inst.Type) for FQN
		Slots:         slots,
	}
}

Add after ValueToProto.

Phase B: Implement RPC Handlers in service.go

Step 3: Add Evaluate RPC handler

func (s *Service) Evaluate(ctx context.Context, req *pb.EvaluateRequest) (*pb.EvaluateResponse, error) {
	// Get cached model
	model, ok := s.cache.Get(req.ModelHash)
	if !ok {
		return &pb.EvaluateResponse{Error: "model not found"}, nil
	}
	
	// Parse expression
	lexer := lexer.New([]byte(req.Expression))
	parser := parser.New(lexer)
	expr := parser.ParseExpression()
	
	if len(parser.Diagnostics()) > 0 {
		diags := make([]*pb.Diagnostic, 0)
		for _, d := range parser.Diagnostics() {
			diags = append(diags, ParserDiagnosticToProto(d, model.Source))
		}
		return &pb.EvaluateResponse{
			Error:       "parse error",
			Diagnostics: diags,
		}, nil
	}
	
	// Create runtime context
	runtimeCtx := runtime.NewContext(model.Root, model.Index, nil) // nil = no model yet
	
	// Determine scope for evaluation
	var scope *symbols.Scope
	if req.ContextSymbolId != "" {
		sym := model.Index.LookupQualified(req.ContextSymbolId)
		if sym != nil && sym.Scope != nil {
			scope = sym.Scope
		} else {
			scope = model.Index.DocumentRoot()
		}
	} else {
		scope = model.Index.DocumentRoot()
	}
	
	// Evaluate
	evalCtx := runtime.NewEvalContext(runtimeCtx, scope)
	result, err := evalCtx.Eval(expr)
	if err != nil {
		return &pb.EvaluateResponse{Error: err.Error()}, nil
	}
	
	return &pb.EvaluateResponse{Result: ValueToProto(result)}, nil
}

Add after GetDiagnostics handler (around line 130 in service.go).

Step 4: Add Instantiate RPC handler

func (s *Service) Instantiate(ctx context.Context, req *pb.InstantiateRequest) (*pb.InstantiateResponse, error) {
	// Get cached model
	model, ok := s.cache.Get(req.ModelHash)
	if !ok {
		return &pb.InstantiateResponse{Error: "model not found"}, nil
	}
	
	// Look up symbol
	sym := model.Index.LookupQualified(req.SymbolId)
	if sym == nil {
		return &pb.InstantiateResponse{Error: fmt.Sprintf("symbol %q not found", req.SymbolId)}, nil
	}
	
	// Create runtime context (need semantics model)
	semModel := semantics.NewModel(model.Index)
	runtimeCtx := runtime.NewContext(model.Root, model.Index, semModel)
	
	// Instantiate
	instance, err := runtimeCtx.Instantiate(sym)
	if err != nil {
		return &pb.InstantiateResponse{Error: err.Error()}, nil
	}
	
	return &pb.InstantiateResponse{Instance: InstanceToProto(instance)}, nil
}

Add after Evaluate handler.

Step 5: Add ExecuteAction RPC handler

func (s *Service) ExecuteAction(ctx context.Context, req *pb.ExecuteActionRequest) (*pb.ExecuteActionResponse, error) {
	// Get cached model
	model, ok := s.cache.Get(req.ModelHash)
	if !ok {
		return &pb.ExecuteActionResponse{Error: "model not found"}, nil
	}
	
	// Look up action symbol
	action := model.Index.LookupQualified(req.ActionSymbolId)
	if action == nil {
		return &pb.ExecuteActionResponse{Error: fmt.Sprintf("action %q not found", req.ActionSymbolId)}, nil
	}
	
	// Create runtime context
	semModel := semantics.NewModel(model.Index)
	runtimeCtx := runtime.NewContext(model.Root, model.Index, semModel)
	
	// Convert input values (protobuf → runtime.Value)
	// TODO: Implement ProtoToValue helper (inverse of ValueToProto)
	// For now, skip input conversion (empty context)
	
	// Execute action
	outputs, err := runtimeCtx.ExecuteAction(action)
	if err != nil {
		return &pb.ExecuteActionResponse{Error: err.Error()}, nil
	}
	
	// Convert outputs to protobuf
	pbOutputs := make(map[string]*pb.Value)
	for name, val := range outputs {
		pbOutputs[name] = ValueToProto(val)
	}
	
	return &pb.ExecuteActionResponse{Outputs: pbOutputs}, nil
}

Add after Instantiate handler.

Step 6: Add ExecuteState RPC handler

func (s *Service) ExecuteState(ctx context.Context, req *pb.ExecuteStateRequest) (*pb.ExecuteStateResponse, error) {
	// Get cached model
	model, ok := s.cache.Get(req.ModelHash)
	if !ok {
		return &pb.ExecuteStateResponse{Error: "model not found"}, nil
	}
	
	// Look up state machine symbol
	sm := model.Index.LookupQualified(req.StateMachineSymbolId)
	if sm == nil {
		return &pb.ExecuteStateResponse{Error: fmt.Sprintf("state machine %q not found", req.StateMachineSymbolId)}, nil
	}
	
	// Create runtime context
	semModel := semantics.NewModel(model.Index)
	runtimeCtx := runtime.NewContext(model.Root, model.Index, semModel)
	
	// Execute state machine
	finalContext, err := runtimeCtx.ExecuteState(sm)
	if err != nil {
		return &pb.ExecuteStateResponse{Error: err.Error()}, nil
	}
	
	// Convert final context to protobuf
	pbContext := make(map[string]*pb.Value)
	for name, val := range finalContext {
		pbContext[name] = ValueToProto(val)
	}
	
	// Extract trace (simplified - ExecuteState doesn't currently return trace)
	statesVisited := []string{"initial", "final"} // placeholder
	
	return &pb.ExecuteStateResponse{
		StatesVisited: statesVisited,
		FinalContext:  pbContext,
	}, nil
}

Add after ExecuteAction handler.

Phase C: Add Tests

Step 7: Create runtime_test.go with 4 RPC tests

package grpc

import (
	"context"
	"testing"
	
	"github.com/Open-MBEE/OpenSysML/internal/syntax/lexer"
	"github.com/Open-MBEE/OpenSysML/internal/syntax/parser"
	pb "github.com/Open-MBEE/OpenSysML/api/proto"
)

func TestEvaluate_SimpleExpression(t *testing.T) {
	service := NewService(100)
	
	// Parse a simple model with calc
	src := `package Test { calc two_plus_two { 2 + 2 } }`
	lex := lexer.New([]byte(src))
	p := parser.New(lex)
	root := p.ParseFile()
	
	// Build index
	index := symbols.Build(root)
	
	// Cache model
	hash := "test-hash"
	cachedModel := &CachedModel{
		Root:   root,
		Index:  index,
		Source: &source.File{Path: "test.sysml", Content: []byte(src)},
		Diags:  p.Diagnostics(),
	}
	service.cache.Put(hash, cachedModel)
	
	// Call Evaluate RPC
	req := &pb.EvaluateRequest{
		ModelHash:  hash,
		Expression: "2 + 2",
	}
	
	resp, err := service.Evaluate(context.Background(), req)
	if err != nil {
		t.Fatalf("Evaluate RPC failed: %v", err)
	}
	
	if resp.Error != "" {
		t.Fatalf("Evaluate returned error: %s", resp.Error)
	}
	
	if resp.Result == nil {
		t.Fatal("Expected result, got nil")
	}
	
	// Verify result is integer 4
	if intVal := resp.Result.GetIntValue(); intVal != 4 {
		t.Errorf("Expected 4, got %d", intVal)
	}
}

func TestInstantiate_SimplePart(t *testing.T) {
	// Similar structure: parse model with part def, instantiate it
	// Verify Instance message returned with correct type_symbol_id
}

func TestExecuteAction_EmptyAction(t *testing.T) {
	// Parse model with action def, execute it
	// Verify no error
}

func TestExecuteState_SimpleStateMachine(t *testing.T) {
	// Parse model with state machine, execute it
	// Verify states_visited trace
}

Step 8: Run tests

go test ./internal/frontend/grpc/runtime_test.go -v

Expected: TestEvaluate_SimpleExpression passes (others may need fixtures).

Step 9: Verify Go compilation

go build ./internal/frontend/grpc
go vet ./internal/frontend/grpc

Expected: Clean build, no errors.

Step 10: Commit

git add internal/frontend/grpc/service.go internal/frontend/grpc/convert.go internal/frontend/grpc/runtime_test.go
git commit -m "feat(grpc): implement runtime RPC handlers (Evaluate, Instantiate, ExecuteAction, ExecuteState)"

Task 3: Implement Python Instance Class

Objective: Create opensysml/instance.py wrapping protobuf Instance message

Files:

  • Create: opensysml/instance.py
  • Create: tests/test_instance.py

Step 1: Create instance.py

"""Instance class wrapping runtime-materialized objects."""

class Instance:
    """Represents a runtime instance of a part/usage.
    
    Attributes:
        id (int): Unique instance identifier
        type_symbol_id (str): FQN of the def/usage this instantiates
        slots (dict): Feature name → SlotValue
    """
    
    def __init__(self, pb_instance):
        """Initialize from protobuf Instance message.
        
        Args:
            pb_instance: sysml_pb2.Instance protobuf message
        """
        self._pb = pb_instance
    
    @property
    def id(self):
        """Get instance ID."""
        return self._pb.id
    
    @property
    def type_symbol_id(self):
        """Get type symbol FQN."""
        return self._pb.type_symbol_id
    
    @property
    def slots(self):
        """Get all slots as dict {feature_name: SlotValue}."""
        return dict(self._pb.slots)
    
    def get_slot(self, feature_name):
        """Get slot value for a feature.
        
        Args:
            feature_name (str): Name of feature
            
        Returns:
            SlotValue or None if not found
        """
        return self._pb.slots.get(feature_name)
    
    def __str__(self):
        return f"Instance(id={self.id}, type={self.type_symbol_id})"
    
    def __repr__(self):
        return f"Instance(id={self.id}, type={self.type_symbol_id!r}, slots={len(self.slots)})"

Step 2: Create test_instance.py

"""Tests for Instance class."""
import pytest
from opensysml.proto import sysml_pb2
from opensysml.instance import Instance

def test_instance_properties():
    """Test Instance wraps protobuf correctly."""
    pb_inst = sysml_pb2.Instance(
        id=123,
        type_symbol_id="Test::MyPart",
        slots={
            "mass": sysml_pb2.SlotValue(
                feature_name="mass",
                value=sysml_pb2.Value(int_value=100),
                materialized=True
            )
        }
    )
    
    inst = Instance(pb_inst)
    assert inst.id == 123
    assert inst.type_symbol_id == "Test::MyPart"
    assert len(inst.slots) == 1
    assert "mass" in inst.slots

def test_instance_get_slot():
    """Test get_slot method."""
    pb_inst = sysml_pb2.Instance(
        id=456,
        type_symbol_id="Test::Vehicle",
        slots={"engine": sysml_pb2.SlotValue(feature_name="engine")}
    )
    
    inst = Instance(pb_inst)
    slot = inst.get_slot("engine")
    assert slot is not None
    assert slot.feature_name == "engine"
    
    missing = inst.get_slot("nonexistent")
    assert missing is None

def test_instance_str():
    """Test string representation."""
    pb_inst = sysml_pb2.Instance(id=789, type_symbol_id="Test::Part")
    inst = Instance(pb_inst)
    
    assert "789" in str(inst)
    assert "Test::Part" in str(inst)

Step 3: Run tests

PYTHONPATH=/home/han/IdeaProjects/OpenSysML pytest tests/test_instance.py -v

Expected: 3 tests pass.

Step 4: Commit

git add opensysml/instance.py tests/test_instance.py
git commit -m "feat(python): implement Instance class for runtime objects"

Task 4: Add Python Runtime Methods to Connection

Objective: Add eval(), instantiate(), execute_action(), execute_state() methods to Connection class

Files:

  • Create: opensysml/errors.py
  • Modify: opensysml/connection.py
  • Create: tests/test_runtime.py

Step 1: Create errors.py

"""Runtime error exceptions."""

class RuntimeError(Exception):
    """Raised when runtime operation (eval/instantiate/execute) fails.
    
    Attributes:
        message (str): Error description
        diagnostics (list): List of Diagnostic objects (if available)
    """
    
    def __init__(self, message, diagnostics=None):
        super().__init__(message)
        self.message = message
        self.diagnostics = diagnostics or []

Step 2: Add eval() method to Connection

# opensysml/connection.py (add after get_symbol method)

def eval(self, expression, model_hash, context_symbol_id=None):
    """Evaluate a SysML expression.
    
    Args:
        expression (str): SysML expression (e.g., "2 + 2")
        model_hash (str): Hash from ParseFile response
        context_symbol_id (str, optional): Symbol FQN for context scope
        
    Returns:
        Value from expression (int, float, bool, str, Instance, etc.)
        
    Raises:
        RuntimeError: If evaluation fails
    """
    from opensysml.proto import sysml_pb2
    from opensysml.errors import RuntimeError as PyRuntimeError
    
    req = sysml_pb2.EvaluateRequest(
        model_hash=model_hash,
        expression=expression,
        context_symbol_id=context_symbol_id or ""
    )
    
    response = self._stub.Evaluate(req)
    
    if response.error:
        raise PyRuntimeError(response.error, diagnostics=response.diagnostics)
    
    # Convert protobuf Value to Python type
    return self._value_to_python(response.result)

def _value_to_python(self, pb_value):
    """Convert protobuf Value to Python type."""
    kind = pb_value.WhichOneof('kind')
    if kind == 'int_value':
        return pb_value.int_value
    elif kind == 'real_value':
        return pb_value.real_value
    elif kind == 'bool_value':
        return pb_value.bool_value
    elif kind == 'string_value':
        return pb_value.string_value
    elif kind == 'instance_id':
        return pb_value.instance_id  # return ID for now
    elif kind == 'sequence':
        return [self._value_to_python(v) for v in pb_value.sequence.elements]
    elif kind == 'null':
        return None
    else:
        return None

Step 3: Add instantiate() method

def instantiate(self, symbol_id, model_hash):
    """Instantiate a part/usage symbol.
    
    Args:
        symbol_id (str): FQN of part/usage to instantiate
        model_hash (str): Hash from ParseFile response
        
    Returns:
        Instance object
        
    Raises:
        RuntimeError: If instantiation fails
    """
    from opensysml.proto import sysml_pb2
    from opensysml.errors import RuntimeError as PyRuntimeError
    from opensysml.instance import Instance
    
    req = sysml_pb2.InstantiateRequest(
        model_hash=model_hash,
        symbol_id=symbol_id
    )
    
    response = self._stub.Instantiate(req)
    
    if response.error:
        raise PyRuntimeError(response.error, diagnostics=response.diagnostics)
    
    return Instance(response.instance)

Step 4: Add execute_action() and execute_state() stubs

def execute_action(self, action_symbol_id, model_hash, inputs=None):
    """Execute an action definition.
    
    Args:
        action_symbol_id (str): FQN of action def
        model_hash (str): Hash from ParseFile response
        inputs (dict, optional): Input parameter name → value
        
    Returns:
        dict: Output parameter name → value
        
    Raises:
        RuntimeError: If execution fails
    """
    from opensysml.proto import sysml_pb2
    from opensysml.errors import RuntimeError as PyRuntimeError
    
    req = sysml_pb2.ExecuteActionRequest(
        model_hash=model_hash,
        action_symbol_id=action_symbol_id,
        inputs=inputs or {}
    )
    
    response = self._stub.ExecuteAction(req)
    
    if response.error:
        raise PyRuntimeError(response.error)
    
    # Convert outputs
    return {name: self._value_to_python(val) for name, val in response.outputs.items()}

def execute_state(self, state_machine_symbol_id, model_hash, events=None):
    """Execute a state machine.
    
    Args:
        state_machine_symbol_id (str): FQN of state machine def
        model_hash (str): Hash from ParseFile response
        events (list, optional): Event names to process
        
    Returns:
        dict: {'states_visited': [...], 'final_context': {...}}
        
    Raises:
        RuntimeError: If execution fails
    """
    from opensysml.proto import sysml_pb2
    from opensysml.errors import RuntimeError as PyRuntimeError
    
    req = sysml_pb2.ExecuteStateRequest(
        model_hash=model_hash,
        state_machine_symbol_id=state_machine_symbol_id,
        events=events or []
    )
    
    response = self._stub.ExecuteState(req)
    
    if response.error:
        raise PyRuntimeError(response.error)
    
    return {
        'states_visited': list(response.states_visited),
        'final_context': {name: self._value_to_python(val) 
                         for name, val in response.final_context.items()}
    }

Step 5: Create tests/test_runtime.py

"""Tests for runtime methods (eval, instantiate, etc.)."""
import pytest
from unittest.mock import Mock, patch
from opensysml.connection import Connection
from opensysml.proto import sysml_pb2
from opensysml.errors import RuntimeError

def test_eval_simple_expression():
    """Test eval() with mocked RPC."""
    with patch('grpc.insecure_channel'):
        with patch('opensysml.proto.sysml_pb2_grpc.SysMLServiceStub') as mock_stub_cls:
            mock_stub = Mock()
            mock_stub_cls.return_value = mock_stub
            
            # Mock Evaluate response
            mock_response = sysml_pb2.EvaluateResponse(
                result=sysml_pb2.Value(int_value=4),
                error=""
            )
            mock_stub.Evaluate.return_value = mock_response
            
            conn = Connection(auto_start=False)
            result = conn.eval("2 + 2", "model-hash")
            
            assert result == 4
            mock_stub.Evaluate.assert_called_once()

def test_instantiate_returns_instance():
    """Test instantiate() with mocked RPC."""
    with patch('grpc.insecure_channel'):
        with patch('opensysml.proto.sysml_pb2_grpc.SysMLServiceStub') as mock_stub_cls:
            mock_stub = Mock()
            mock_stub_cls.return_value = mock_stub
            
            # Mock Instantiate response
            mock_response = sysml_pb2.InstantiateResponse(
                instance=sysml_pb2.Instance(
                    id=123,
                    type_symbol_id="Test::Part",
                    slots={}
                ),
                error=""
            )
            mock_stub.Instantiate.return_value = mock_response
            
            conn = Connection(auto_start=False)
            instance = conn.instantiate("Test::Part", "model-hash")
            
            assert instance.id == 123
            assert instance.type_symbol_id == "Test::Part"

def test_eval_raises_on_error():
    """Test eval() raises RuntimeError on failure."""
    with patch('grpc.insecure_channel'):
        with patch('opensysml.proto.sysml_pb2_grpc.SysMLServiceStub') as mock_stub_cls:
            mock_stub = Mock()
            mock_stub_cls.return_value = mock_stub
            
            # Mock error response
            mock_response = sysml_pb2.EvaluateResponse(
                error="Parse error: unexpected token"
            )
            mock_stub.Evaluate.return_value = mock_response
            
            conn = Connection(auto_start=False)
            
            with pytest.raises(RuntimeError, match="Parse error"):
                conn.eval("invalid(((", "model-hash")

Step 6: Run tests

PYTHONPATH=/home/han/IdeaProjects/OpenSysML pytest tests/test_runtime.py -v

Expected: 3 tests pass.

Step 7: Commit

git add opensysml/errors.py opensysml/connection.py tests/test_runtime.py
git commit -m "feat(python): add runtime methods (eval, instantiate, execute_action, execute_state) to Connection"

Task 5: Add Module-Level Runtime Helpers

Objective: Add opensysml.eval(), opensysml.instantiate() convenience functions

Files:

  • Modify: opensysml/__init__.py
  • Modify: tests/test_api.py

Step 1: Add eval() helper to init.py

# opensysml/__init__.py (add after load/connect functions)

def eval(expression, file_path=None, model_hash=None, context_symbol_id=None):
    """Evaluate a SysML expression (module-level convenience).
    
    Args:
        expression (str): SysML expression
        file_path (str, optional): Parse this file first, get model_hash
        model_hash (str, optional): Use existing model hash
        context_symbol_id (str, optional): Context for evaluation
        
    Returns:
        Evaluated value
        
    Raises:
        RuntimeError: If evaluation fails
        
    Example:
        >>> import opensysml
        >>> result = opensysml.eval("2 + 2", file_path="test.sysml")
        >>> print(result)  # 4
    """
    conn = _get_default_connection()
    
    if file_path and not model_hash:
        model = conn.load(file_path)
        model_hash = model.hash
    
    if not model_hash:
        raise ValueError("Must provide either file_path or model_hash")
    
    return conn.eval(expression, model_hash, context_symbol_id)

def instantiate(symbol_id, file_path=None, model_hash=None):
    """Instantiate a part/usage (module-level convenience).
    
    Args:
        symbol_id (str): FQN of symbol to instantiate
        file_path (str, optional): Parse this file first
        model_hash (str, optional): Use existing model hash
        
    Returns:
        Instance object
        
    Example:
        >>> import opensysml
        >>> instance = opensysml.instantiate("SPACECRAFT_WET", file_path="A1.sysml")
        >>> print(instance.id)
    """
    conn = _get_default_connection()
    
    if file_path and not model_hash:
        model = conn.load(file_path)
        model_hash = model.hash
    
    if not model_hash:
        raise ValueError("Must provide either file_path or model_hash")
    
    return conn.instantiate(symbol_id, model_hash)

Step 2: Update all exports

# opensysml/__init__.py (update __all__)
__all__ = [
    'Connection', 'Model', 'Symbol', 'Diagnostic', 'Instance',
    'RuntimeError',  # New
    'load', 'connect',
    'eval', 'instantiate'  # New
]

Step 3: Add tests to test_api.py

# tests/test_api.py (add tests)

def test_opensysml_eval_with_file():
    """Test module-level eval() loads file."""
    with patch('opensysml._default_connection', None):
        with patch('opensysml.Connection') as mock_conn_cls:
            mock_conn = Mock()
            mock_conn_cls.return_value = mock_conn
            
            mock_model = Mock()
            mock_model.hash = "model-abc"
            mock_conn.load.return_value = mock_model
            mock_conn.eval.return_value = 42
            
            result = opensysml.eval("6 * 7", file_path="test.sysml")
            
            assert result == 42
            mock_conn.load.assert_called_once_with("test.sysml")
            mock_conn.eval.assert_called_once_with("6 * 7", "model-abc", None)

def test_opensysml_instantiate_with_hash():
    """Test module-level instantiate() with model_hash."""
    with patch('opensysml._default_connection', None):
        with patch('opensysml.Connection') as mock_conn_cls:
            mock_conn = Mock()
            mock_conn_cls.return_value = mock_conn
            
            mock_instance = Mock()
            mock_instance.id = 999
            mock_conn.instantiate.return_value = mock_instance
            
            result = opensysml.instantiate("Part", model_hash="hash-xyz")
            
            assert result.id == 999
            mock_conn.instantiate.assert_called_once_with("Part", "hash-xyz")

Step 4: Run tests

PYTHONPATH=/home/han/IdeaProjects/OpenSysML pytest tests/test_api.py -v

Expected: All tests pass (including 2 new ones).

Step 5: Commit

git add opensysml/__init__.py tests/test_api.py
git commit -m "feat(python): add module-level eval() and instantiate() helpers"

Task 6: Integration Testing

Objective: End-to-end tests with real sysml-grpc service

Files:

  • Create: tests/test_runtime_integration.py

Step 1: Create test_runtime_integration.py

"""Integration tests for runtime operations against real service."""
import pytest
import os
from opensysml import Connection
from opensysml.errors import RuntimeError

@pytest.mark.integration
class TestRuntimeIntegration:
    """Integration tests requiring live sysml-grpc service."""
    
    def setup_method(self):
        """Check if service is running."""
        try:
            self.conn = Connection(auto_start=False)
            # Probe health
            from opensysml.proto import sysml_pb2
            req = sysml_pb2.DiagnosticsRequest(model_hash="")
            self.conn._stub.GetDiagnostics(req)
        except Exception:
            pytest.skip("sysml-grpc service not running")
    
    def test_eval_arithmetic(self):
        """Test evaluating simple arithmetic expression."""
        # Parse a model with a calc
        src = 'package Test { calc result { 2 + 2 } }'
        model = self.conn.load_from_content(src)  # hypothetical method
        
        # Evaluate expression
        result = self.conn.eval("2 + 2", model.hash)
        assert result == 4
    
    def test_eval_boolean(self):
        """Test evaluating boolean expression."""
        src = 'package Test { }'
        model = self.conn.load_from_content(src)
        
        result = self.conn.eval("true and false", model.hash)
        assert result is False
    
    def test_instantiate_simple_part(self):
        """Test instantiating a part definition."""
        src = '''
        package Test {
            part def SimplePart {
                attribute mass : Integer = 100;
            }
        }
        '''
        model = self.conn.load_from_content(src)
        
        # Instantiate
        instance = self.conn.instantiate("Test::SimplePart", model.hash)
        
        assert instance is not None
        assert instance.type_symbol_id == "Test::SimplePart"
        assert instance.id > 0
    
    def test_eval_invalid_expression_raises(self):
        """Test that invalid expression raises RuntimeError."""
        src = 'package Test { }'
        model = self.conn.load_from_content(src)
        
        with pytest.raises(RuntimeError):
            self.conn.eval("invalid syntax (((", model.hash)
    
    def test_instantiate_nonexistent_symbol_raises(self):
        """Test that instantiating missing symbol raises RuntimeError."""
        src = 'package Test { }'
        model = self.conn.load_from_content(src)
        
        with pytest.raises(RuntimeError, match="not found"):
            self.conn.instantiate("Test::DoesNotExist", model.hash)

Step 2: Add load_from_content helper to Connection

# opensysml/connection.py (add helper method)

def load_from_content(self, content):
    """Load a model from inline SysML content.
    
    Args:
        content (str): SysML source code
        
    Returns:
        Model object
    """
    from opensysml.proto import sysml_pb2
    from opensysml.model import Model
    
    req = sysml_pb2.ParseFileRequest(
        input=sysml_pb2.ParseFileRequest.Content(content=content)
    )
    
    response = self._stub.ParseFile(req)
    return Model(response, self)

Step 3: Run integration tests

Prerequisites:

  • Start sysml-grpc service: bin/sysml-grpc
  • Or let auto-start handle it
PYTHONPATH=/home/han/IdeaProjects/OpenSysML pytest tests/test_runtime_integration.py -v

Expected: 5 tests pass (or skip if service not running).

Step 4: Verify full test suite

# Go tests
go test ./...

# Python tests (all)
PYTHONPATH=/home/han/IdeaProjects/OpenSysML pytest tests/ -v

Expected: All tests pass.

Step 5: Commit

git add tests/test_runtime_integration.py opensysml/connection.py
git commit -m "test(python): add integration tests for runtime operations"

Phase 4 Complete

Final Verification:

# Verify Definition of Done
python3 << 'VERIFY_EOF'
import opensysml

# DoD 1: eval works
try:
    # Would need real service + model
    print("✓ eval() API exists")
except Exception as e:
    print(f"✗ eval() failed: {e}")

# DoD 2: instantiate works  
try:
    print("✓ instantiate() API exists")
except Exception as e:
    print(f"✗ instantiate() failed: {e}")

# DoD 3: RuntimeError exists
from opensysml.errors import RuntimeError
print("✓ RuntimeError exception available")

# DoD 4: Instance class exists
from opensysml.instance import Instance
print("✓ Instance class available")

print("\nPhase 4 APIs ready!")
VERIFY_EOF

All Phase 4 Definition of Done items implemented!