Motivation
Knowledge Work (KW) is the backbone of modern society, driving innovative work across diverse sectors [e.g., 1, 8, 15]. It involves two core, interconnected processes, (1) sensemaking: The active process where individuals seek, interpret information; and (2) production of artifacts: The creation phase, where individuals leverage information, data, and ideas to generate new outputs, such as solutions, designs, and knowledge products [4]. Together, they encompass a wide array of high-value activities, including learning, planning, writing, programming, creativity, design, problem-solving, to name a few [e.g., 2, 3, 4, 15, 18].
Challenge: Situated and Complex Knowledge Work
KW is inherently challenging due to its situated and dynamic nature [1, 8, 15]. Knowledge tasks are deeply embedded within the complex, context-dependent lives and circumstances of users. Unlike routine processes, KW involves ill-defined tasks that continuously evolve as new information surfaces and needs shift, demanding constant human-technology adaptation. A key challenge is designing intelligent and customizable authoring techniques that can flexibly serve the diverse needs, expertise, and environments of individual knowledge workers.
Opportunity: SLM / LLM Agents
Recently, Generative AI (GAI) —specifically Large and Small Language Models (LLMs / SLMs) —has emerged with the potential to fundamentally reshape knowledge work [5, 6]. We propose to leverage the power of GAI by examining the implications of visual authoring tools [9] powered by GAI Agents in supporting KW. Our objective, as illustrated conceptually in Figure 1, is to integrate multi-agent collaboration with visual authoring to support high-value daily KW activities, with application in content creation (i.e., create ideas and artifacts) and information tasks (e.g., search, learn, analyze, summarize) [e.g., 15, 17].
Scientific Locks
One major limitation of current L/SLMs is their difficulty in reliably decomposing complex, ill-defined knowledge-work tasks into meaningful and goal-aligned sub-tasks. Task decomposition is context-dependent, varies across domains, and often fails to reflect the user’s actual objectives, constraints, or working practices. As a result, sub-tasks generated solely by LLMs may be poorly scoped, misaligned with user intent, or inefficiently distributed across agents.
• To address this challenge, this thesis proposes to combine Multi-Agent LLM systems (MALLMs) [11] with Knowledge Graphs (KGs) [e.g., 19] to support adaptive task decomposition, agent coordination, and explainable decision-making in situated knowledge work.
• KGs will serve as the semantic backbone of the proposed system. They will provide a structured, explicit, and interpretable representation of key elements involved in knowledge work, including: goals, sub-goals, tasks, and processes; agents and their competencies (i.e., skills); resources and generated artifacts; and user profiles and contextual constraints.
• The KG also acts as a shared memory and coordination space for multi-agent collaboration. Agents continuously query and update the KG to: monitor task progress, adapt decisions based on new information; resolve dependencies or conflicts; and maintain a consistent global view of the project state.
• In addition to orchestration and reasoning, KGs will support Retrieval-Augmented Generation (RAG) [e.g., 19] by structuring and indexing domain knowledge, resources, and previously generated artifacts. This allows agents to: retrieve semantically relevant information; ground generation in explicit knowledge; and reduce hallucinations and improve consistency.
• Using GAI Agents offers significant advantages over single LLM use [11]. Most notably, agents can leverage specialized collective intelligence, profiles, and skills, closely mirroring the natural human dynamic of cooperative work. By combining specialized GAI Agents for specific sub-tasks, we expect to achieve performance improvements in complex KW.
Figure 1: Overview of Our Proposed Approach of Authoring Tools For Knowledge Work —Multi-Agent LLM Systems
Through this proposal we aim to leverage and extend our existing conceptual framework [16] for supporting knowledge workers. This framework is defined by the intersection of three axes:
• Generative Multimodal Authoring: The ability to create various types of rich content (text, image, code, diagrams, infographics, slides, podcasts, advanced explanations, etc.) using GAI Agents.
• Visual Design: The use of visuo-spatial interactions to structure and manipulate content.
• Information Processing: The capacity for AI agents to support users to analyze, synthesize, and make sense of complex information.
Furthermore, we aim to build upon and leverage our previous research into process structuring techniques [10] —methods for organizing complex, multi-step collaborative tasks. Moreover, this research will be conducted as part of the Pinfig authoring tool [12], which is developing a set of visual authoring tools powered by Multi-agent systems to support specific KW tasks. Pinfig is already home to specialized tools, demonstrating the viability of our approach. For instance, we are now leveraging multi-agent LLM systems coupled with visual authoring to support specific tasks of content creation (cf., AgentCollab, AgentPods, see appendix) and information tasks (cf., AgentCanvas, see appendix). This integration allows us to transform multi-step KW tasks into manageable, collaborative processes guided by specialized GAI Agents within a visual workspace. The Ph.D. candidate can leverage and extend the authoring and multi-agent LLM techniques designed in the research tool Pinfig [12].
Research Questions (RQs)
This thesis sets out to address the following RQs to guide the study and design of GAI multi-agent visual authoring techniques to support KW:
• RQ1 (Needs & Design): What are the needs of knowledge workers regarding visual authoring tools powered by GAI Agents to support their daily activities (e.g., content creation, information, and educational tasks)? Furthermore, how can we effectively identify and integrate these needs using a participatory design approach?
• RQ2 (Capabilities & Architecture): What are the optimal capabilities, architectures, and visual authoring techniques for GAI Agent tools that can effectively support the core KW processes of integrating, sensemaking, production of artifacts, and authoring various types of original and multimodal contents? How does the KG support adaptive task decomposition and help assigning agents to specific tasks?
• RQ3 (Integration & Responsibilities): How can GAI agent authoring tools be effectively integrated into the daily workflows and tasks of knowledge workers? What are the practical advantages and challenges, and how can we safeguard and promote responsible factors such as privacy, trust, independence, and ethics in their deployment?
Methodology and Evaluation
Our research will employ Design-Based Research (DBR) framework [13, 14]. DBR is an iterative, cyclical methodology well-suited for developing and evaluating innovative systems within real-world, complex contexts —a necessity for addressing the situated nature of KW.
We will conduct research across diverse contexts —including local collaborators like Lannion’s Fablab, Anticipa, IUT Lannion, and schools —to ensure the transferability and real-world relevance of our approach. Our DBR cycle comprises three main phases, ensuring the tools we develop are aligned with the practical needs of knowledge workers:
Phase 1: Needs Elicitation & Participatory Design
• Goal: To establish a clear understanding of user needs (RQ1).
• Activity: We will conduct participatory design workshops involving target end-users. These workshops will be used to systematically collect user needs, define specific KW tasks, generate initial design ideas, and perform early-stage usability testing of low-fidelity concepts.
• Outcome: A validated set of requirements and design principles for the visual authoring tools.
Phase 2: Design and Development
• Goal: To translate requirements into functional systems (RQ2).
• Activity: We will conduct iterative design, prototyping, and development of the visual authoring tools powered by our GAI Agents (e.g., refining AgentCanvas and AgentCollab within the Pinfig ecosystem).
• Outcome: High-fidelity, functional prototypes ready for real-world testing.
Phase 3: In-Situ Pilot Studies and Evaluation
• Goal: To measure the impact and identify challenges (RQ3).
• Activity: We will deploy the prototypes in in-situ pilot studies (testing in the users' actual working environment). Evaluation will use mixed methods (quantitative performance metrics and qualitative feedback) to assess the benefits, challenges, and responsible factors (e.g., trust, independence) of the tools in daily KW workflows.
• Outcome: Empirical evidence and refined models for integrating GAI Agents into KW.
Main Research Contributions
This thesis aims to extend prior research and establish a foundational understanding of GAI Agent-powered visual authoring tools through three main, interconnected contributions:
• Conceptual Models (Theoretical contribution): We will formulate the core notions and models underpinning visual authoring powered by multi-agents for KW. This includes defining the architectural principles, interaction paradigms, and conceptual framework that govern how collaborative AI agents facilitate human sensemaking and artifact production.
• Interaction Techniques (Technical contribution): We will design and implement new visual authoring techniques powered by multi-agents to specifically support complex, situated KW tasks. This involves introducing novel interaction models and visual languages that allow users to effectively manage and guide the collective intelligence of the GAI Agent team.
• Empirical Insights (Empirical contribution): We will provide evidence of the tangible benefits of visual authoring techniques powered by multi-agents for KW. This empirical validation will quantify the impact on key performance indicators (e.g., productivity, quality of artifacts [e.g. 10]) and offer critical insights into the real-world advantages and challenges.
[1] Drucker, P. F. (1999). Knowledge-worker productivity: The biggest challenge. California Management Review, 41(2), 79–94.
[2] Dervin, B. (1998). Sense-making theory and practice: An overview of user interests in knowledge seeking and use. Journal of Knowledge Management, 2(2), 36–46.
[3] Klein, G., Moon, B., & Hoffman, R. R. (2006). Making sense of sensemaking: Alternative perspectives. IEEE Intelligent Systems, 21(4), 70–73.
[4] Kim, Y. H., Choe, E. K., Lee, B., & Seo, J. (2019). Understanding personal productivity: How knowledge workers define, evaluate, and reflect on their productivity. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–12). ACM.
[5] Sætra, H. S. (2023). Generative AI: Here to stay, but for good? Technology in Society, 75, 102372. https://doi.org/10.1016/j.techsoc.2023.102372
[6] Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790
[7] Gero, K. I., Long, T., & Chilton, L. B. (2023). Social dynamics of AI support in creative writing. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (pp. 1–15). ACM. https://doi.org/10.1145/3544548.3580782
[8] Woodruff, A., et al. (2024). How knowledge workers think generative AI will (not) transform their industries. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. ACM.
[9] Cypher, A., et al. (2010). No code required: Giving users tools to transform the web. Morgan Kaufmann.
[10] Ez-Zaouia, M., & Carrillo, R. (2024). The group folding effect: The role of collaborative process structuring and social interaction in group work. ACM Transactions on Computer-Human Interaction, 31(2), Article 15. https://doi.org/10.1145/3622783
[11] Guo, T., et al. (2024). Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI-24) (pp. 8048–8057). https://doi.org/10.24963/ijcai.2024/890
[12] Ez-Zaouia, M. (n.d.). Pinfig – Brainstorm. Learn. Write. Collaborate. Visually. https://pinfig.com/
[13] Barab, S., & Squire, K. (2016). Design-based research: Putting a stake in the ground. In Design-based research (pp. 1–14). Psychology Press.
[14] Zimmerman, J., Forlizzi, J., & Evenson, S. (2007). Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 493–502). ACM.
[15] Brachman, M., El-Ashry, A., Dugan, C., & Geyer, W. (2024). How knowledge workers use and want to use LLMs in an enterprise context. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1–8). ACM.
[16] Ez-Zaouia, M., & Boumarafi, Y. (2025). CODIGEN: Enhancing personalized programming education with visual authoring and LLMs. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2025).
[17] Lyu, Y., Zhang, H., Niu, S., & Cai, J. (2024). A preliminary exploration of YouTubers’ use of generative AI in content creation. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1–7). ACM.
[18] Hu, D., Bhuiyan, M. M., Lim, S., Wiese, J., & Lee, S. W. (2024). Unpacking task management tools, values, and worker dynamics. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work (pp. 1–16).
[19] Zhu, X., Xie, Y., Liu, Y., Li, Y., & Hu, W. (2025). Knowledge graph-guided retrieval augmented generation. arXiv preprint arXiv:2502.06864.