Today we had a successful 8th IEEE Int’l Workshop on Next Level of Test Automation (#NEXTA2025). Highlights included:
✅ KEYNOTE by Serge Demeyer
✅ TALKS
- Influence of Pure & Unit-Like Tests on SBFL
- Witness Test Program Generation
- Adaptive Testing for LLM-Based Applications
✅ DISCUSSION!
SEER Lab’s Jeremy Bradbury & Riddhi More are in Naples, Italy 🇮🇹 attending the 18th IEEE Int’l Conference on Software Testing, Verification and Validation (#ICST2025) where they are presenting two research papers. Dr. Bradbury is also a member of the faculty panel at the ICST Doctoral Symposium.
Looking for evidence of data leakage in the #HumanEval code generation #LLM benchmark? Check out our analysis of a subset of HumanEval tasks vs comparable tasks on #ChatGPT, #Claude and #Llama. @riddhimore.bsky.social
Riddhi More presenting at FAIRNESS 2025
Yesterday Riddhi More presented our paper “Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models” at the 1st Int’l Workshop on Fairness in Software Systems (#FAIRNESS2025) in Montreal, Canada. 🇨🇦
Paper preprint, slides & code/data available at: bit.ly/4brEBJm
MSc student Riddhi More & Prof. Jeremy Bradbury are in Montreal, PQ this week attending the IEEE Int’l Conference on Software Analysis, Evolution and Reengineering (#SANER2025) and the 1st Int’l Workshop on Fairness in Software Systems (#FAIRNESS2025).
Dr. Michael Miljanovic presenting at SIGCSE 2025.
Today Dr. Michael Miljanovic presented our latest paper at #SIGCSE2025. The paper titled “How Effective and Efficient Are Student-Written Software Tests?”, is co-authored by Amanda Showler, Michael Miljanovic, and Jeremy Bradbury. 📄: www.sqrlab.ca/publications...
SEER Lab’s Stacey Koornneef and Prof. Michael Miljanovic are in Pittsburgh, PA this week attending the SIGCSE Technical Symposium on Computer Science Education (#SIGCSE2025).
Thank you to the organizers of the #AI Meets Software Quality Int’l Colloquium for inviting Riddhi More & I to speak on Monday. If you're interested in #LLMs & #flakytests the preprints of the papers presented are available at bit.ly/40OvYUa (FlakyXbert) & bit.ly/4jILzNT (augmentation bias).
They will discuss recent research on comparing flaky test detection and classification using few-shot learning versus fine tuning as well as the issue of data augmentation-induced bias in testing datasets.
Tomorrow Prof. Jeremy Bradbury & MSc student Riddhi More will be presenting "FlakyXbert: A Few-Shot Learning Framework for Detecting and Classifying Flaky Tests" at the AI Meets Software Quality International Colloquium (bit.ly/42DAZ4M)
#AI #LLM #testing #flakytests #dataaugmentation #bias
Our paper "Addressing #DataLeakage in #HumanEval Using Combinatorial Test Design" has been accepted for publication at the Int’l Conference on Software Testing, Verification & Validation (#ICST2025), Short Papers, Vision & Emerging Results.
co-author: Riddhi More
arxiv.org/abs/2412.01526
#benchmark
Are you working on #softwaretesting & passionate about test #automation and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation #NEXTA2025, co-located w/ #ICST2025
Submission Date: Jan 10, 2025 [UPDATED!]
bit.ly/nexta2025
I'm happy to share our paper "Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models," has been accepted to the 1st International Workshop on Fairness in Software Systems (FAIRNESS 2025), co-located with #SANER2025! #ontariotech @riddhimore.bsky.social
Happy holidays from the SEER Lab! ❄️
I’m excited to share that our paper "An Analysis of #LLM Fine-Tuning & Few-Shot Learning for Flaky Test Detection & Classification" has been accepted for publication at the Int’l Conf on Software Testing, Verification & Validation (#ICST2025). This paper is first-authored by @riddhimore.bsky.social
I had a great time today attending the #OntarioTech Cakes & #AI Winter Show-And-Tell event. A great opportunity to engage with researchers from across the university and establish interdisciplinary collaborations. Thanks to @petelewis.bsky.social for organizing! 🍰
Interested in joining the Software Engineering & Education Research Lab? Find out more at bit.ly/3ARpCe8 #OntarioTech 🇨🇦
Last night I participated in a graduate student panel and a poster session at @ontariotechu.bsky.social! It was great to be able to network with everyone and see the research coming from other graduate students!
I'm happy to share that w/ Pooria Madani, Khalil El-Khatib & Natalija Vlajic we have been awarded a $382,000 grant from the National Cybersecurity Consortium to study "Transformative Adversaries: Leveraging Generative Pretrained Transformers for the Dev. of Next-Gen. Metamorphic Malware Engines"
SEER Lab on the Ontario Tech University campus.
Did you know that the Software Engineering & Education Research Lab is located on Ontario Tech University’s state-of-the-art campus in Oshawa, ON, Canada? 🇨🇦
[photo credit: @ontariotechu.bsky.social]
Prof. Pooria Madani presenting.
I was happy to have Prof. Pooria Madani give a guest lecture today in my Applications of LLMs to Software Engineering graduate course. He presented “Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats” — arxiv.org/abs/2410.22293
#OntarioTech @pooriamadani.bsky.social
New paper alert! 📃 ⏰
How Effective and Efficient Are Student-Written Software Tests?
- Amanda Showler, Michael Miljanovic, Jeremy Bradbury
Proc. of the Technical Symposium on Computer Science Education (SIGCSE 2025) [to appear]
Preprint: bit.ly/sigcse2025te...
#softwaretesting #CSEd #OntarioTech
NEXTA 2025 Call for Papers
Are you working on #softwaretesting & passionate about test #automation, test generation, and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation - #NEXTA2025, co-located with #ICST2025
Submission Date: Jan 3, 2025
More Info: bit.ly/nexta2025
Members of the SEER Lab at Ontario Tech University.
I’m happy to now be on 🦋! 👋🏻
I’m a CS professor at Ontario Tech University in Oshawa, Canada. 🇨🇦 At #OntarioTech, I lead the Software Engineering & Education Lab (www.seerlab.ca) where we research applications of #AI, #NLP, #LLMs to #softwareengineering, #softwaretesting & #CSEd.