Blazers can be casual or they can be formal. One thing that can make them look more dressy is an emblem or crest. It may seem like a detail, but it’s the kind of thing that can make quite a difference. Perhaps this is because we associate these crested jackets with institutions of higher learning, yacht clubs, or other members-only organizations. Still you can dress-down any formal blazer if you pair it with casual pants or even jeans.
Two very nice models, IMHO:
Blazers with emblems pictured:
Left: Abercrombie Men’s Lake Arnold Blazer, navy
Right: Ralph Lauren Women’s Cotton Knit Crest Blazer Sweater Jacket, navy