This country was founded for the protection of Life, Liberty, and the pursuit of happiness. This means the protection of property rights. With out property rights Liberty and the right to keep what one has earned is meaningless. Restaurants are private property, so is the food bought by the restaurant owner, the owner has the right to dis-own what he owns for what ever consideration he chooses to accept. If the government says the owner has to dis-own his property in a transaction the owner of the property chooses not to accept, the government is no longer protecting Liberty, but confiscating property.

Make no mistake about it. A restaurant owner that refuses to conduct a property transaction with some one based on something as trivial as a persons skin color, is a jerk, and disserving of a boycott by other restaurant goers.