AI Powered Knowledge Management System

Team

• E/19/004, R.B.ABEYSINGHE, email
• E/19/096, E.M.C.Y.B.EKANAYAKE, email
• E/19/100, E.P.S.G.ELLAWALA, email

Supervisors

• Dr. Asitha Bandaranayake, email
• Prof Roshan Ragel, email

Table of content

1. Abstract
2. Related works
3. Methodology
4. Experiment Setup and Implementation
5. Results and Analysis
6. Conclusion
7. Publications
8. Links

Abstract

Leveraging AI-Powered Knowledge Management Systems to Enhance Operational Efficiency

National Research and Education Networks (NRENs) play a crucial role in supporting academic and research communities by providing advanced technological infrastructure and services. As these organizations grow in complexity, there is an increasing need for efficient knowledge management systems to support their operations.

This research project aims to explore the potential of AI-powered knowledge management systems in enhancing the operational efficiency of NRENs, with a focus on managing:

• Institutional knowledge
• Training materials
• Administrative guidelines

Related works

Methodology

Research Objectives

• Primary Goal: Develop an AI-powered knowledge management system tailored for NREN operations
• Focus Areas: Institutional knowledge preservation, training material organization, administrative guideline accessibility
• Target Outcome: Enhanced operational efficiency through intelligent information retrieval and management

Experiment Setup and Implementation

Technical Architecture

RAG (Retrieval-Augmented Generation) System

Our approach leverages RAG architecture to combine the benefits of large language models with domain-specific knowledge retrieval:

Core Components

1. Document Processing Pipeline

   • Text extraction from various formats (PDF, DOC, HTML)
   • Chunking and preprocessing
   • Vector embedding generation

2. Vector Database

   • Semantic search capabilities
   • Efficient similarity matching
   • Scalable storage for large document collections

3. Language Model Integration

   • Context-aware response generation
   • Query understanding and refinement
   • Multi-turn conversation support

Results and Analysis

Current Progress

• RAG Architecture Study: Comprehensive analysis of Retrieval-Augmented Generation systems
• Vector Embeddings Research: Deep dive into semantic search and similarity matching techniques
• Prototype Development: Created small-scale chatbots for concept validation
• Component Integration: Successfully tested retrieval and generation workflows

Conclusion

This research demonstrates the viability and advantages of AI-powered knowledge management systems in supporting the evolving needs of NRENs. By automating and enhancing knowledge retrieval and accessibility, such systems can significantly boost operational efficiency.

Publications

Links

• Project Repository
• Project Page
• Department of Computer Engineering
• University of Peradeniya

Technology Stack

Language Models

• Primary Requirement: GPT-3.5 Turbo or GPT-4
• Use Cases:
  • Text generation and summarization
  • Query understanding and response synthesis
  • Context-aware information retrieval

Vector Processing

• Embedding Models: OpenAI text-embedding-ada-002 or similar
• Vector Database: Pinecone, Weaviate, or Chroma
• Similarity Search: Cosine similarity, semantic matching

Development Framework

- Python 3.8+
- LangChain for LLM orchestration
- OpenAI API for language models
- Vector database (Pinecone/Weaviate)
- FastAPI for backend services
- React/Next.js for frontend interface