Named after the song by J Ralph. There are two Kansas Cities, and while one is in Kansas, the larger, more well-known one is actually just across the river in Missouri. Somebody unfamiliar with the region might go to the smaller one when they really wanted to go to the larger one, because they made the assumption that "Kansas City is obviously in Kansas". The song uses this confusion as an analogy for the way a mark thinks when they're being conned by a Kansas City Shuffle-type con.
You look left and they fall right
Into the Kansas City Shuffle
Its a they-think you-think you don't know
Type of Kansas City hustle
Where you take your time
Wait your turn
And hang them up, and out to dry
Experiment is finished.