Advances in Machine Learning & Artificial Intelligence(AMLAI)

Manager Data Science, Publicis Sapient, India

Publications

• Research Article   
  Cost-Efficient Secure Hybrid RAG Assessment
  Author(s): Jitender Singh*
  
  This paper presents a cost-effective and scalable hybrid methodology for evaluating retrieval-augmented generation (RAG) systems using specialized pretrained models and advanced metrics, designed for critical domains like healthcare and finance. LLM judge-based evaluation approaches are hindered by significant score inconsistencies across identical input runs (up to 35% variations) and high computational costs while traditional NLP approaches, overly reliant on entity or phrase matching, lack a multi-faceted perspective and fail to capture deeper semantic understanding. Our methodology addresses these challenges with a novel approach that combines an ensemble of fine-tuned pretrained models and advanced NLP techniques, ensuring consistent, reproducible evaluations across multiple dimensions, while being tailored for offline scenarios to eliminate reliance on internet-connected systems.. Read More»
  

  Abstract HTML PDF