I won't reveal too much of the story, but in the face of imminent apocalypse (due to an alien invasion that slowly renders all electrical devices useless) two of the characters come to some interesting conclusions:
George shook his head slowly, in wonder. He said, "Streetcars and buses, ocean liners-Pete, this means we’re going back to the original source of horsepower. Horses. If you want to invest, buy horses. Particularly mares. A brood mare is going to be worth a thousand times her weight in platinum."
"Right. But don’t forget steam. We’ll still have steam engines, stationary and locomotive."
"Sure, that’s right. The iron horse again, for the long hauls. But Dobbin for the short ones. Can you ride, Peter?"
"Used to, but I think I’m getting too old. I’ll settle for a bicycle. Say, better buy a bike first thing tomorrow before the run on them starts. I know I’m going to."
"Good tip. And I used to be a good bike rider. It’ll be swell with no autos around to louse you up."
And later, after some time has passed:
"How’s New York?”
“Fine, George. Down to its last million people, and stabilizing there. No crowding and plenty of room for everybody. The air-why, it’s better than Atlantic City, without gasoline fumes.”
“Enough horses to go around yet?”
“Almost. But bicycling’s the craze; the factories can’t turn out enough to meet the demand. There’s a cycling club in almost every block and all the able-bodied cycle to and from work. Doing ‘em good, too; a few more years and the doctors will go on short rations.”
“You got a bike?”
“Sure, a pre-vader one. Average five miles a day on it, and I eat like a horse.”
Another quasi-post-apocalyptic short story that at least begins with a bicycle is "It's a Good Life" by Jerome Bixby. Since there are no "external" inputs of fossil fuels the first character we're introduced to is delivering groceries by bike.
For a good cyberpunk short story check out "Bicycle Repairman" by Bruce Sterling.