Anomaly detection aims to identify rare and scarce anomalies, which is particularly challenging in computational pathology, where disease-related data are often limited or nonexistent. Existing anomaly detection methods, primarily designed for industrial settings, face limitations in pathology due to computational constraints, diverse tissue structures, and lack of interpretability. To address these challenges, we propose Ano-NAViLa, a normal and abnormal pathology knowledge-augmented vision-language model for anomaly detection in pathology images. Ano-NAViLa utilizes a pre-trained vision-language model with a lightweight trainable MLP, facilitating computationally efficiency. By incorporating both normal and abnormal pathology knowledge, Ano-NAViLa enhances accuracy and robustness to variability in pathology images and provides interpretability through image-text associations. Evaluated on two lymph node datasets from different organs, Ano-NAViLa achieves the state-of-the-art performance in anomaly detection and localization, outperforming competing models.