This attitude prevailed in the mid '40s and early '50s...
...when jet engines were first contemplated for use in commercial passenger aircraft. Jet engines at the time were extremely fuel-hungry and unreliable. Over time, these problems were resolved. The same may or may not be true of scram jet technology.
It's true that the Concorde was a commercial failure because R&D, construction and operating costs were never justified by the limited number of people able and willing to pay for what it would ultimately cost to travel at supersonic speed. This is likely true for hypersonic travel as well, but that doesn't mean that it's not worth experimenting with the technology.