Ya you can totally get away with it. You don't have to wear a button up all the time. They look good but sometimes it good to do something a little different with your style. Hope I helped. ;-)
I normally wear button ups, but I found this shirt in my brother's closet and I like it.. Just feel like I'm not the type that can get away with just a shirt...