Ornstein studied acting as a child at the American Academy of Dramatic Arts in New York City, and later studied under Stella Adler. After high school, he enrolled in the Mason Gross School of the Arts. He quit after his first year and moved to New York City to begin working professionally.

Ornstein's first film credit was in Crossing Delancey, where he played the minor character "Mickey". Television credits include a recurring role as Detective Bonnaventura on Homicide: Life on the Street, and a two-episode arc on Third Watch, as well as parts in Seinfeld, Law & Order, Law & Order: Special Victims Unit, and Law & Order: Criminal Intent. Ornstein's stage acting credits span twenty-five years worth of new plays in New York City, in addition to creating the role of Louis Ironson in the 1991 World Premiere of Angels in America at the Eureka Theater in San Francisco. Ornstein plays Chuck Marstein on Sons of Anarchy.