Perhaps we should be concerned about the fact that the resources of the species seem to be increasingly turned to the construction of extravagant machines for cheating on homework and winning raffles. There are already tried-and-true ways of accomplishing these things.
A set of logic gates can be used to process a proposition, i.e., it can tell you what you want to know about a certain proposition by decomposing, processing, and then re-composing the thought. But what we want to know, phrased in terms of a discursive proposition, is not necessarily always the real answer, or the necessary knowledge.
A bit of AI sitting in the middle of a field would harm no one, of course. The problem is not the thing itself as a substance or entity. The danger is in making the existing mechanisms of the world subject to AI control. And since we don't really have a handle on what the mechanism of the industrialized world is, we fear losing control of it. It's like fearing the ability to make a Frankenstein without understanding what he is being made out of, and what form he might take. The Chinese box itself is simply a remarkable automated deterministic abacus; the real difficulty is that people want to use it to control their machines, as they might hitch their wagons to a wild horse, or delegate their roulette bets to a trained monkey.