ICCMA
International Competition on Computational Models of Argumentation
Home Competition 2015 Competition 2017 Competition 2019 Competition 2021 Competition 2023 Competition 2025 Important Dates Calls Tracks Rules Execution environment Submissions Solvers Organization Contact

ICCMA 2025 Competition Tracks

ICCMA 2025 consists of four tracks:

  • main track
  • approximate track
  • dynamic track
  • ABA track

Each track is composed of multiple sub-tracks, defined by a combination of a reasoning problem and an argumentation semantics.

Argumentation systems (solvers) can be submitted for evaluation into any choice of sub-tracks, i.e., there is no requirement to support e.g. all semantics for a specific reasoning problem, or all reasoning problems for a specific semantics.

NEW FOR 2025: For each (sub-)track (except No-Limits Track), an argumentation system can additionally take part in a fixed-SAT solver comparison. A participating solver is then ranked also in separate list with only other solvers taking part in this comparison (in addition to the normal ranking). Note the special requirements for taking part of this comparison.

NEW FOR 2025: In ICCMA 2025 we narrowed the list of semantics to some degree. Note that the stage semantics is not planned to be part of ICCMA 2025.

Main Track

Note: The focus of the Main Track is to evaluate sequential core argumentation reasoning engines available in open source. Systems combining different core reasoning engines e.g. via portfolio-style techniques, systems employing parallel computations via the use of multiple processor cores, as well as systems which will not be made available in open source are invited to the special No-Limits Track which consists of the same subtracks as the Main Track. The organizers reserve the right to move Main Track submissions to No-Limits based on case-by-case analysis.

Problem Setting

We recall the definition of Dung's Abstract Argumentation Frameworks (AFs) [Dung 95] and its semantics.

An AF is a directed graph F = (A,R) where A is a set of arguments, and R ⊆ A × A is the attack relation. For a,b ∈ A, we say that a attacks b when (a,b) ∈ R. If in addition b attacks c ∈ A, then a defends c against b. The same concepts are extended to sets of arguments: S ⊆ A attacks (respectively defends) an argument b ∈ A if there is some a ∈ S that attacks (respectively defends) b.

The relevant semantics of AFs are defined as follows. Let the range of S ⊆ A be S {a ∈ A | S attacks a}. A set S ⊆ A is

  • conflict-free (S ∈ CF(F)) if S does not attack any of its elements;
  • admissible (S ∈ AD(F)) if S ∈ CF(F) and S defends all its elements against all their attackers.
  • a complete extension (S ∈ CO(F)) if S ∈ AD(F) and S contains all the arguments that it defends;
  • a preferred extension (S ∈ PR(F)) if S is a maximal (w.r.t. set inclusion) admissible extension;
  • a stable extension (S ∈ ST(F)) if S ∈ CF(F) and S attacks each argument not in S;
  • a semi-stable extension (S ∈ SST(F)) if S is complete and has a minimal range (w.r.t. set inclusion) among the complete extensions;
  • an ideal extension (S ∈ ID(F)) if S is the maximal (w.r.t. set inclusion) admissible extension that is a subset of each preferred extension.

Subtracks

The semantics included in the Main Track are complete (CO), preferred (PR), stable (ST), semi-stable (SST), and ideal (ID).

The reasoning problems included in the Main Track are

  • DC-σ: Given F=(A,R) and a ∈ A, decide whether a is credulously accepted under σ and if it is, report a σ-extension that contains a. An argument a ∈ A is credulously accepted under σ if a is in at least one σ-extension.
  • DS-σ: Given F=(A,R) and a ∈ A, decide whether a is skeptically accepted under σ and if it is not, report a σ-extension that does not contain a. An argument a ∈ A is skeptically accepted under σ if a is in all σ-extensions.
  • SE-σ: Given F=(A,R), return a σ-extension of F.
Each of the following combination of semantics and reasoning mode is a subtrack:

  • DC-{CO|ST|SST}
  • DS-{PR|ST|SST}
  • SE-{PR|ST|SST|ID}

Ranking

Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.

No-Limits Track

The No-Limits Track is a more permissible version of the Main Track and has the same problems as subtracks. In particular, solvers are allowed to run on multiple cores and can be portfolio-based (i.e. combine the usage of several solvers). The ranking is otherwise the same as for the Main Track, but wall-clock time is used instead of CPU time.

Approximate Track

Problem Setting

Abstract argumentation, as defined above. Correctness requirements and ranking are different than other tracks: incorrect solutions are simply discarded and only the number of correct solutions is taken into account. The time limit for the track is lower than for the other tracks, namely 60 seconds CPU time per instance.

Subtracks

Semantics: CO, PR, ST, SST, and ID. Reasoning modes: DC-σ and DS-σ. Each of the following combination of semantics and reasoning mode is a subtrack:

  • DC-{CO|ST|SST|ID}
  • DS-{PR|ST|SST}

Ranking

Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The solver with the largest amount of correctly solved instances within a time limit of 60 seconds wins. If needed, cumulative CPU running time over solved instances is used as a tie-breaker.

Dynamic Track

Problem Setting

Abstract argumentation, as defined above. Dynamic changes to an initial AF and acceptance queries are issued by different applications via IPAFAIR, an API for incremental reasoning in abstract argumentation. Please see ipafair.py in the repository for more details.

Subtracks

The subtracks in the Dynamic Track are DC-CO, DS-PR, DC-ST, and DS-ST. Each subtrack consists of applications which call an AF solver which implements the interface, applying changes to the underlying AF between solver calls. The name of the subtrack determines the allowed reasoning task and semantics. For example, in the DS-PR subtrack, the AF solver is initialized using the preferred semantics, and only solve_skept() calls are allowed with a single query argument. Both the query argument and the underlying AF may change between these calls.

Ranking

Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.

ABA Track

Problem Setting

Assumption-based Argumentation (ABA) [Bondarenko et al 97] and the corresponding semantics are defined as follows.

An ABA framework is a tuple F = (L,R,A,‾) where

  • L is a set of literals
  • R is a set of rules
  • A ⊆ L is a (non-empty) set of assumptions
  • ‾ is a mapping from assumptions to literals, that represents a notion of contrariness
A rule h ← b1,...,bn is made of literals h, b1,... bn ∈ L, and n ≥ 0. We call h the head of the rule and b1,... bn the body. A rule without a body (i.e. h ←, or alternatively h ←⊤) is a fact. We consider only flat ABA frameworks, i.e. ABA frameworks where the head of a rule is never an assumption.

A sentence a ∈ L is derivable from a set X ⊆ A via rules R, denoted by X ⊢ a, if a ∈ X or there is a sequence of rules (r1,...,rn) such that head(rn) = a and for each rule ri we have ri ∈ R and each sentence in the body of ri is derived from rules earlier in the sequence or in X. A set of assumptions A1 attacks a set of assumptions A2 if the contrary of some a ∈ A2 is derivable from A1. A set of assumptions A1 defends an assumption a if A1 attacks every set of assumption that attacks a.

The semantics for an ABA framework F can be defined as follows. Given a set of assumptions X ⊆ A,

  • X ∈ CF(F) if it does not attack any of its elements;
  • X ∈ AD(F) if X ∈ CF(F) and X defends all its elements;
  • X ∈ CO(F) if X ∈ AD(F) and X contains every assumption it defends;
  • X ∈ PR(F) if it is a ⊆-maximal admissible set of F;
  • X ∈ ST(F) if X ∈ CF(F) and X attacks every assumption not in X.
We call a set of assumptions in σ(F) a σ-extension for semantics σ ∈ {CO, PR, ST} (complete, preferred and stable, respectively).

Subtracks

Semantics: CO, PR, ST.

The reasoning tasks for the ABA Track are

  • DC-σ: Given ABF F=(L,R,A,‾) and s ∈ L, decide whether s is credulously accepted under σ.
  • DS-σ: Given ABF F=(L,R,A,‾) and s ∈ L, decide whether s is skeptically accepted under σ.
  • SE-σ: Given ABF F=(L,R,A,‾), return a σ-extension of F.

Each of the following combination of semantics and reasoning mode is a subtrack:

  • DC-{CO|ST}
  • DS-{PR|ST}
  • SE-{PR|ST}

Ranking

Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.

Fixed-SAT-Solver Comparison

Note: after discussion at SAFA 2024, the rules of this comparison will be adapted - stay tuned!

Any solver can be marked for taking part in the Fixed-SAT-Solver comparison. Then the solver will be evaluated normally and, additionally, for each (sub-)track (except No-Limit) all participating solvers in the Fixed-SAT-Solver comparison will be compared separately. This is meant to evaluate solvers relying heavily on SAT solvers in a more uniform comparison. Otherwise, the choice of different SAT solvers might lead to differences in performance, making fair comparisons of such solvers difficult.

If a solver takes part in the Fixed-SAT-Solver comparison, the solver must implement the IPASIR interface for incremental SAT solvers. We will provide an example later on. The solver will then interact with a SAT solver only with the IPASIR interface and be compiled with a specific SAT solver for ICCMA 2025. Taking part in the Fixed-SAT-Solver comparison excludes then use of any other search engine similar to SAT solvers (like answer set programming). Usage of a SAT solver is limited to usage of the IPASIR interface (i.e., the solver may only use the interface to delegate computationally intenstive computations). The organizers reserve the right to remove submissions to the Fixed-SAT-Solver comparison based on a case-by-case analysis.

References

[Dung 95] P. M. Dung, On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games. Artif. Intell. 77(2): 321-358 (1995)

[Bondarenko et al 97] A. Bondarenko, P. M. Dung, R. Kowalski, F. Toni, An Abstract, Argumentation-Theoretic Approach to Default Reasoning. Artif. Intell. 93: 63-101 (1997)




Terms