Charles Stross outlines the reason SF writers pretty much stopped writing short stories en masse in the mid-to-late 1950s:
A typical modern novel is in the range 85,000-140,000 words. But there’s nothing inevitable about this. The shortest work of fiction I ever wrote and sold was seven words long; the longest was 196,000 words. I’ve written plenty of short stories, in the 3000-8000 word range, novelettes (8000-18,000 words), and novellas (20,000-45,000 words). (Anything longer than a novella is a “short novel” and deeply unfashionable these days, at least in adult genre fiction, which seems to be sold by the kilogram.)
[…]
Genre science fiction in the US literary tradition has its roots in the era of the pulp magazines, from roughly 1920 to roughly 1955. (The British SF/F field evolved similarly, so I’m going to use the US field as my reference point.) These were the main supply of mass-market fiction to the general public in the days before television, when reading a short story was a viable form of mass entertainment, and consequently there was a relatively fertile market for short fiction up to novella length. In addition, many of these magazines serialized novels: it was as serials that Isaac Asimov’s Foundation and E. E. “Doc” Smith’s The Skylark of Space were originally published, among others.
For a while, during this period, it was possible to earn a living (not a very good living) churning out pulp fiction in short formats. It’s how Robert Heinlein supplemented his navy pension in the 1930s; it’s how many of the later-great authors first gained their audiences. But it was never a good living, and in the 1950s the bottom fell out of the pulp market — the distribution channel itself largely dried up and blew away, a victim of structural inefficiencies and competition from other entertainment media. The number of SF titles on sale crashed, and the number of copies each sold also crashed. Luckily for the writers a new medium was emerging: the mass market paperback, distributed via the same wholesale channel as the pulp magazines and sold through supermarkets and drugstore wire-racks. These paperbacks were typically short by modern standards: in some cases they provided a market for novellas (25,000 words and up — Ace Doubles consisted of two novellas, printed and bound back-to-back and upside-down relative to one another, making a single book).
The market for short fiction gradually recovered somewhat. In addition to the surviving SF magazines (now repackaged as digest-format paperback monthlies) anthologies emerged as a market. But after 1955 it was never again truly possible to earn a living writing short stories (although this may be changing thanks to the e-publishing format shift — it’s increasingly possible to publish stand-alone shorter works, or to start up a curatorial e-periodical or “web magazine” as the hip young folks call them). And the readership profile of the remaining magazines slowly began to creep upwards, as new readers discovered SF via the paperback book rather than the pulp magazine. With this upward trending demographic profile, the SF magazines entered a protracted, generational spiral of dwindling sales: today they still exist, but nobody would call a US newsstand magazine with monthly sales of 10,000-15,000 copies a success story.
A side-effect of dwindling sales is that the fixed overheads of running a magazine (the editor’s pay check) remains the same but there’s less money to go around. Consequently, pay rates for short fiction stagnated from the late 1950s onwards. 2 cents/word was a decent wage in 1955 — it was $20 for a thousand words, so $80-500 for a short story or novelette. But the monthly magazines were still paying 5 cents/word in the late 1990s! This was pin money. It was a symbolic reward. It would cover your postage and office supplies bill — if you were frugal.
