How to add / remove class in jQuery dynamically on click?

You can simply use this jquery to add/remove class on click of anchor tag

$('#MainMenu li a').on('click', function(){
    $('#MainMenu li a.active').removeClass('active');
    $(this).addClass('active');
});

Take a look this sample fiddle: https://jsfiddle.net/ym9Lb7t3/

If you don't want to remove class from previously selected anchor, use .toggleClass()

$('#MainMenu li a').on('click', function () {
    $(this).toggleClass('active')
});

If you want to toggle class, or remove class you can also do it using Javascript ( without jquery)

Toggle Class JS based

<button class="red btn" type="button" id="btn"  onclick="changeColor()">Click Me to toggle class</button>

Javascript

function changeColor() {
     document.getElementById("btn").classList.toggle('red');
     document.getElementById("btn").classList.toggle('green');
}

Replace/Add Class using Pure Javascript

 function changeColor() {
const btn = document.querySelector('#btn');

// print existing classes
console.log(btn.className); // 

// add new class
btn.className += 'btn';

// replace all existing classes
btn.className = 'red btn';
}

You can check detailed article here "Toggle or Replace Class (Javascript based, without jQuery)"