Task-Skill Mapping

Formal Definition

Task-Skill Mapping defines the relationship between tasks and the skills required to perform them. It provides the foundation for evaluating agent capabilities and determining skill requirements.

Task Definition

A task is formally defined as: $\mathcal{T}=⟨description,{\mathcal{R}}_{\mathcal{T}},{\theta }_{\mathcal{T}}⟩$

Where:

• $description\in {\Sigma }^{\ast }$: Natural language description of the task
• ${\mathcal{R}}_{\mathcal{T}}\subseteq 2^{\mathcal{S}}$: Set of required skills
• ${\theta }_{\mathcal{T}}$: Task-specific parameters and constraints

Skill Requirements

For a task $\mathcal{T}$, the required skills ${\mathcal{R}}_{\mathcal{T}}$ represent the capabilities an agent needs to successfully complete the task.

Agent Skill Set

An agent $A$ has a skill set: ${\mathcal{S}}_A\subseteq \mathcal{S}$

Fitness Function for Task-Skill Mapping

The agent-level fitness function evaluates how well an agent can perform a task:

$\Phi (\mathcal{T},A)=\frac{|{\mathcal{R}}_{\mathcal{T}}\cap {\mathcal{S}}_A|}{|{\mathcal{R}}_{\mathcal{T}}|}\cdot {\prod }_{S\in {\mathcal{R}}_{\mathcal{T}}\cap {\mathcal{S}}_A}{\varphi }_S(\mathcal{T})$

Components:

1. Coverage Term: $\frac{|{\mathcal{R}}_{\mathcal{T}}\cap {\mathcal{S}}_A|}{|{\mathcal{R}}_{\mathcal{T}}|}$

   • Proportion of required skills the agent possesses
   • Ranges from 0 (no required skills) to 1 (all required skills)

2. Quality Term: ${\prod }_{S\in {\mathcal{R}}_{\mathcal{T}}\cap {\mathcal{S}}_A}{\varphi }_S(\mathcal{T})$

   • Product of individual skill fitness values
   • Ensures all skills must perform reasonably well

Key Properties and Characteristics

1. Completeness Requirement

For perfect performance, an agent needs all required skills: $\Phi (\mathcal{T},A)=1⟺{\mathcal{R}}_{\mathcal{T}}\subseteq {\mathcal{S}}_A\wedge \forall S\in {\mathcal{R}}_{\mathcal{T}},{\varphi }_S(\mathcal{T})=1$

2. Acceptable Performance

Reasonable performance requires a threshold: $\Phi (\mathcal{T},A)\ge {\tau }_{acceptable}$

Where ${\tau }_{acceptable}$ is a task-specific or domain-specific threshold.

3. Skill Gap Analysis

The skill gap is: $\text{gap}(\mathcal{T},A)={\mathcal{R}}_{\mathcal{T}}\setminus {\mathcal{S}}_A$

These are skills the agent lacks but needs for the task.

4. Redundant Skills

Skills the agent has but doesn’t need: $\text{redundant}(A,\mathcal{T})={\mathcal{S}}_A\setminus {\mathcal{R}}_{\mathcal{T}}$

Research Context and Applications

Task-skill mapping enables:

• Capability Assessment: Determining if an agent can perform a task
• Skill Gap Analysis: Identifying missing capabilities
• Learning Prioritization: Focusing on high-impact skills
• Task Assignment: Matching tasks to capable agents
• Team Formation: Assembling agents with complementary skills

In LLM research:

• Determining which model to use for specific tasks
• Identifying areas for fine-tuning
• Evaluating multi-task performance
• Designing benchmark suites
• Understanding capability limitations

Mapping Strategies

1. Explicit Mapping

Manually define required skills for each task: ${\mathcal{R}}_{\mathcal{T}}=\{{\mathcal{S}}_1,{\mathcal{S}}_2,{\mathcal{S}}_3\}$

2. Learned Mapping

Learn skill requirements from examples: ${\mathcal{R}}_{\mathcal{T}}=f_{learn}(\{({\mathcal{T}}_i,{\mathcal{S}}_i)\})$

3. Hierarchical Mapping

Use skill hierarchy to infer requirements: $\mathcal{S}\in {\mathcal{R}}_{\mathcal{T}}⟹\mathcal{P}(\mathcal{S})\subseteq {\mathcal{R}}_{\mathcal{T}}$

Connections to Other Concepts

• Skills (𝒮): Elements being mapped to tasks
• Fitness Functions (Φ, φ): Evaluate mapping quality
• Skill Hierarchy: Determines prerequisite requirements
• Metaskills: Analysis metaskills perform task-skill mapping
• Agent Capabilities: Evaluated through task-skill mapping

Task Taxonomy

Tasks can be categorized by skill requirements:

1. Simple Tasks

Few required skills, straightforward mapping: $|{\mathcal{R}}_{\mathcal{T}}|\le k_{simple}$

2. Complex Tasks

Many required skills, intricate dependencies: $|{\mathcal{R}}_{\mathcal{T}}|>k_{complex}$

3. Composite Tasks

Can be decomposed into subtasks: $\mathcal{T}={\mathcal{T}}_1\oplus {\mathcal{T}}_2\oplus \cdots \oplus {\mathcal{T}}_n$

Open Research Questions

1. Automatic Skill Extraction: How to automatically identify ${\mathcal{R}}_{\mathcal{T}}$ from task descriptions?

2. Minimal Skills: What is the minimal set of skills sufficient for a task?

3. Skill Sufficiency: When are additional skills beyond ${\mathcal{R}}_{\mathcal{T}}$ beneficial?

4. Dynamic Requirements: How do skill requirements change with context?

5. Granularity: At what level of granularity should skills be defined for effective mapping?

6. Uncertainty: How to handle uncertainty in skill requirements?

7. Transfer Mapping: How do task-skill mappings transfer across domains?

8. Emergent Requirements: Can task requirements include emergent superskills not explicitly listed?