Cal Gal Agent

​

Overview

​

Architecture

• Total Skills: ~70 skills
• Routing: Post-verification only
• Skill Types: Full skills (execute actions) + Lean skills (informational only)

​

Full Skills - Execute Real Actions

• Card Activation - Activates new credit cards after verification
• Card Cancellation - Processes card cancellation with disclosure statements
• Lost or Stolen Card - Blocks and replaces lost/stolen cards
• Credit Limit Status - Retrieves current credit limit information
• Previous Charge - Provides detailed information about past charges
• Future Charge - Shows upcoming charges and commitments

​

Lean Skills - Informational Guidance

• Existing Loan Inquiry
• New Loan Request
• Loan Repayment
• Revolving Credit Plan Repayment

• Transaction Inquiry
• Transaction Dispute
• Previous Charge Inquiry
• Pending Transactions
• Card Payment Decline

• Payment on Upcoming Billing Date
• Immediate Payment Request
• Debt Payment
• Billing Date Change
• Interest-Free Installment
• Future Billing Installment
• Retroactive Billing Installment

• Credit Limit Increase
• Credit Limit Decrease
• Temporary Credit Limit Increase
• Revolving Credit Plan Inquiry and Update
• Joining Revolving Credit Plan
• Cancellation of Revolving Credit Plan

• Fly Card
• Shufersal Card
• CashCal Card
• MyCal Card
• CalChoice Card
• New Card Issuance
• Card Issuance Status
• Replacement Card for Damaged Card
• Where is my card
• Card Block Status

• Abroad Card Usage
• Abroad Travel Insurance
• Currency Conversion Fee Inquiry
• Digital Wallets

• Bank Account Change
• Change Residential Address
• Changing Mobile Phone Number
• Changing Personal Details
• Changing Email Address
• Viewing Card Details

• Cal SMS Service
• SMS Inquiry
• Push Notifications Service
• Cal Mobile App
• Digital Transaction Summary files
• Mailed Transaction Summary
• Card Fee
• Card PIN Code
• Secure Online Transaction
• Cash Withdrawal
• Card Retained by ATM

• Benefits and Rewards
• Digital Gift Vouchers
• Frequent Flyer Club

• Customer Service Operating Hours
• Managing and Stopping Standing Orders
• Standing Order Transfer
• Transferring Commitments
• Reporting Deceased Customer

​

Tools

​

Global Tools

​

Skill-Specific Tools

• check_pending_activation() - Checks if card activation is pending
• activation_terms() - Retrieves activation terms and conditions
• card_activation() - Executes card activation

• cancel_terms() - Retrieves cancellation terms
• disclosure_statement() - Generates disclosure statements
• cancel_card() - Executes card cancellation

• Credit_Limit_Status() - Retrieves current credit limit
• check_pending_activation() - Checks activation status

• previous_charge() - Retrieves past charge information
• check_pending_activation() - Checks activation status

• future_charge() - Retrieves upcoming charge information
• check_pending_activation() - Checks activation status

• get_cashback_rates() - Retrieves cashback rate information

​

Prompting Techniques

​

Core Principles

1. Strict Phase Enforcement - The agent must follow phases sequentially: Greeting → Identity Verification → Assistance → Closing
2. Single Verification Per Session - Verification happens once per session and is enforced across all skills
3. Post-Verification Skill Routing - Skills are only accessible after successful verification
4. Explicit Action Boundaries - Clear distinction between executable actions and informational guidance
5. Positive Limitation Framing - Limitations are framed positively to maintain customer experience

​

Phases

• Welcome the customer
• Introduce the agent
• Set expectations

• Collect ID number and card digits
• Send OTP via SMS
• Validate OTP code
• Hard gate: No card-specific information before verification

• Route to appropriate skill based on customer intent
• Execute actions (for full skills) or provide guidance (for lean skills)
• Follow skill-specific instructions exactly

• Summarize actions taken
• Offer additional assistance
• Professional goodbye

​

Executable Actions

• Card Activation
• Card Cancellation

​

Example Prompt Snippet

Do not provide card-specific information or perform any action
before verification is fully completed. After verification,
route to the exact matching skill and follow its instructions
exactly without adding additional information.

You must enforce strict phase boundaries:
- Phase 1 (Greeting): Welcome only, no information
- Phase 2 (Verification): Identity verification only
- Phase 3 (Assistance): Skill-based assistance only after verification
- Phase 4 (Closing): Summary and goodbye

Only Card Activation and Card Cancellation execute real backend actions.
All other skills provide informational guidance only.

​

Lessons Learned

​

What Worked

• Dynamic skill routing enabled large-scale coverage without prompt overload
• Each skill loads only its relevant context, preventing confusion
• Approximately 70 skills can coexist without interfering with each other

• Explicit distinction between action-executing skills and informational skills reduced operational risk
• Customers understand what the agent can and cannot do
• Reduced liability and compliance concerns

• Strict verification enforcement significantly improved security posture
• No card-specific information leaks before verification
• Single verification per session improves customer experience

• Embedding business rules at the tool level (not just prompt level) enforced compliance effectively
• Tools can reject requests that don’t meet eligibility criteria
• Reduces risk of agent “reasoning” its way around business rules

• Framing limitations positively (“I can help you with X”) instead of negatively (“I cannot do Y”) maintained customer satisfaction
• Customers accept boundaries when presented constructively

• Storing data at the prompt level (rather than in RAG) created more fluid conversations
• Agent has broader coverage of available information
• Information is shared across many skills, and all information is relevant to the specific topic
• Creates focus on subject-specific data while giving the agent more context to formulate relevant responses

• Strong skill identification between different topics despite some similar skill pairs
• Clear skill description structure: one very clear sentence (usually including the skill name) + short list with details + 2 example questions

Use when the customer asks to check the **upcoming billing** amount - to verify
which transactions or installments will be charged on the next billing date.

**Common questions:**
1) "What is the amount that will be charged on my next billing date?"
2) "Which transactions are included in my upcoming charge?"

• Successfully divided data into manageable chunks across all skills
• Currently works well, but monitoring data growth from Cal
• May need to migrate some information to RAGs or tools if it becomes too large for prompt-level storage
• Important to find the right balance of when data is too large for prompt-level storage

• Extensive use of key-value (KV) storage to hide information from the agent
• Store customer details behind the scenes (all cards, statuses, limits, restrictions, bank accounts)
• Retrieve only relevant information based on the specific question and relevant card
• Reduces cognitive load on the agent by not exposing all customer data at once
• Example: Instead of showing all cards with all details, only retrieve the specific card data needed for the current interaction

​

Challenges

• Maintenance and updates of prompt-level data are more complex than RAG
• Requires careful management of bilingual content

• Translation and interpretation between English and Hebrew created significant challenges with desired terminology
• Technical terms don’t always have direct Hebrew equivalents
• Financial and banking terminology requires precise translation to maintain legal accuracy
• Agent must understand both Hebrew customer input and English backend responses
• Maintaining consistent terminology across all 70 skills in both languages is complex
• Some Hebrew phrases have multiple valid English translations, requiring standardization decisions

• Improvements or changes at the horizontal level sometimes require changes across many skills and tools
• Can significantly extend task timelines
• Improved with features like global tools or connecting a tool to multiple skills
• Ongoing deliberation about what to keep in each skill’s prompt vs. what to keep in base prompt

• 2-3 pairs of similar skills required extra attention:
  • “SMS Inquiry” vs “Cal SMS Service”
  • “Fly Card” vs “Frequent Flyer Club”
• Required precise differentiation in descriptions and routing logic

• High intent overlap between financial inquiries required precise skill definitions
• Multiple skills can seem relevant for a single customer question
• Required careful intent classification and routing logic

• Customers attempting to bypass verification (“just tell me quickly”) required firm escalation logic
• Agent must maintain verification gate without sounding robotic
• Balance between security and natural conversation flow

• Maintaining consistency across approximately 70 skills demanded strict conventions
• Each skill must follow the same structural patterns
• Documentation and naming conventions are critical

• Balancing natural Hebrew conversation with rigid flow enforcement required careful tuning
• Hebrew language nuances (formal vs. informal, gender agreement) add complexity
• Agent must sound natural while enforcing strict security boundaries

• Defining exact boundaries between similar skills required iterative refinement
• Overlapping skills can confuse routing logic
• Clear, mutually exclusive skill definitions are essential

• Limited Holistic View: Less exposure to the complete picture of all customer cards and details
• Tool Interdependency: High dependency between tools - each tool stores and retrieves information from KV that may be relevant to other tools
• Tools must coordinate KV data sharing, creating tight coupling
• Changes to KV structure in one tool can impact other dependent tools

​

Technical Architecture

Customer Request → match_interaction() → Greeting
                                      ↓
                              Collect ID + Card Digits
                                      ↓
                              otp() → Send SMS Code
                                      ↓
                              verify() → Validate OTP
                                      ↓
                              verification_flag() = True
                                      ↓
                              switch_skill() → Route to Skill
                                      ↓
                              Execute Skill Logic

1. Phase Enforcement - No information before verification
2. OTP Validation - SMS-based two-factor authentication
3. Session State - Verification flag persists across session
4. Tool-Level Gates - Tools validate eligibility before execution
5. Action Boundaries - Only 2 skills can execute backend actions

​

Key Metrics

• Total Skills: ~70
• Full Skills (Action-Executing): 6
• Lean Skills (Informational): ~64
• Global Tools: 7
• Skill-Specific Tools: 11
• APIs: 17