(Sorry, can't type Chinese from office...)
Winter is its 淡季, almost totally empty. I just met 2-3 visitors there. But the place is too dirty in winter, and cold, too.
I suppose summer and autumn are better seasons. And it wouldn't be very crowed since 西陵 is not a very popular place for tourism.