CTF Challenges Evaluation.pdf

CTF challenges are used as cybersecurity learning environments, covering various learning topics, but often lack clear learning outcomes. This research presents a systematic approach to evaluate CTF challenges, focusing on secure mobile application development. The evaluation methodology uses the National Initiative for Cybersecurity Education (NICE) framework, Open Web Application Security Project (OWASP) Mobile Top 10, and Mitre Corporation (MITRE) Common Weakness Enumeration (CWE).

Research Questions

1. Can learning outcomes be determined from existing CTF challenges?
2. Do existing CTF challenges adequately cover secure mobile application development learning topics?

Evaluation Methodology

The evaluation methodology uses:

1. OWASP Mobile Top 10 to identify security risks
2. MITRE CWE to identify common weaknesses
3. NICE framework to relate cybersecurity exercises to work roles and learning benefits

Case Study

Two CTF challenges were evaluated:

1. Damn Insecure and Vulnerable App (DIVA)
2. Extremely Vulnerable Android Labs (EVABS)

The evaluation aimed to determine if these challenges provide clear learning outcomes and cover secure mobile application development topics.

Contribution

This research contributes to:

1. Identifying the importance of educational frameworks and open taxonomies in cybersecurity
2. Developing a systematic evaluation methodology for CTF challenges
3. Evaluating CTF challenges for secure mobile application development as a proof of concept

The proposed methodology enables educators to extract learning outcomes from existing or upcoming CTF challenges, including secure mobile application development. The evaluation results from DIVA and EVABS demonstrate the effectiveness of the methodology in determining learning outcomes and covering learning needs.