While Rue is technically an herb, I think a lot of folks would call it an annoyance. The leaves can irritate the skin of sensitive persons.
With such a reputation, I felt the deer and rabbits would surely leave it alone, but I was wrong. When I transplanted a number of seedlings to an un-fenced planting bed, they were decimated in short time. I do keep this plant in the back of the flower bed to minimize my own brushes with it. There is an aroma when the leaves are brushed, which is distinct, but not near as pungent as say Santolina.
Personally, I think this perennial is a very pretty plant, it does not deserve the bad rap that some gardeners give it. Just because it has an unusual scent and perhaps bothers the skin of some persons – this is not poison oak! I may watch myself, and not rub it all over my body, but it can have its own space. The small, delicate leaves and flowers are a strong contrast to everything else that grows around it. It can stay in my garden.