In the context of drug development, the efficacy of drugs designed using biologics depends on the strength of the interactions between the biologics and the target molecule. In practise, these strengths of the interactions are measured using binding affinity and thus, understanding the binding affinity between an engineered antibody and its target antigen is a critical aspect of antibody engineering as it reflects the overall effectiveness of the antibody in inhibiting the activity of the target antigen. In theory, potential candidate antibodies have a higher binding affinity towards the target antigen. Currently, techniques such as Molecular docking and Molecular dynamics are utilized in quantifying the binding affinity. However, owing to the computational complexity of the aforementioned techniques, running simulations for large antibodies/antigens remains a daunting task. Despite the commendable improvements in deep learningbased binding affinity prediction, such approaches are highly dependent on the quality of the antibody-antigen structures and they tend to overlook the importance of capturing the evolutionary details of proteins upon mutation. To circumvent the said complexities, we propose a deep geometric neural network comprising a structure-based model and a sequence-based model that considers both atomistic and evolutionary details when predicting the binding affinity. The proposed framework exhibited a 10% improvement in mean absolute error compared to the state-of-the-art models while exhibiting a very strong correlation between the predictions and target values.