People sometimes assert that the power to add justices to the Supreme Court was part of the Framers’ scheme of checks and balances, a solution when incumbent justices act against the public interest. In a sense, it doesn’t matter what the Framers had in mind; the Constitution’s language contains no limits on or requirements for changing the size of the Court, so Congress can do what it wants regardless of what the Framers may have thought. But for what it’s worth, while packing and shrinking the Court have occasionally been used (or proposed) for political reasons, it’s unlikely that this is actually what the Framers intended.
In the Federalist Papers that deal with the Supreme Court, Hamilton’s main worry was the difficulty of attracting skilled men to such an unimportant job. He justifies lifetime tenure and a guarantee against reductions in salary on the grounds that no good lawyer would take the position otherwise. Indeed, before the Civil War half a dozen Supreme Court nominees declined to accept their nominations, and the first chief justice, John Jay, resigned in frustration to become governor of New York, having decided just four cases in six years. So it seems unlikely that the Framers thought the remedy for bad judges would be more judges. Impeachment seemed a much better solution.
One reason the Supreme Court was far from a dream job for lawyers was that in the Court’s early decades, the justices had to ride circuit — travel by horse to their assigned area of the country and preside over federal cases there, often holding court in taverns, which were the only places of public assembly in many parts. The original Supreme Court had six justices because the nation was divided into six circuits. When the Seventh Circuit was added, in 1807, a justice was added to the Court (for a total of seven), and when the Eighth and Ninth were added, in 1837, two more justices were added (for a total of nine). After that, Congress’s expansion of the lower federal courts reduced the burden of circuit riding, and by the time the Tenth Circuit was established, in 1929, it was no longer part of a Supreme Court justice’s duties.
So the original Constitution did not fix the size of the Supreme Court for the same reason it did not fix the size of the House of Representatives: Flexibility was necessary to accommodate a growing nation. Again, this does not impose any restrictions on Congress’s power to add members to the Court; but if someone suggests that partisan court-packing was part of the Framers’ original design, they’re probably wrong.