In metallurgy, grain size is an indicator of many different characteristics of the piece of alloy someone will be working on.
In a very general way for copper alloys, the smaller the grains found in the alloy will directly correlate to its ability to be worked under a hammer. The smaller the grains the easier a time one will have trying to forge a piece of copper or brass to shape.
Copper purchased from an extruded source, the most common form, will have a grain size of about 0.05mm. This kind of grain size allows for the grains to slide past each and to be deformed easily without damaging the overall structure.
0.05mm is tiny! The smallest grain size that the Gem Alloys have is 1mm, 20x larger at minimum. Most Gem Alloys have an average grain size of about 5mm, 100x larger with the largest of the Gem Alloys having an average grain size of roughly a full centimeter, 200x larger.
These numbers might not seem particularly extreme but metals are incredibly sensitive to change. And with the grains being as big as they are in the gem alloys even and uniform deformation is made problematic. The larger grains have much less options to slide and move around so breaking occurs.
In addition to this unfortunate bit of metallurgy Gem Alloys and a great deal of other cast copper-based alloys deal with an issue called dendritic coring.
This is the main reason why our ancestors had to cast their bronze tools and weapons as opposed to working them on an anvil.
The bronze our ancestors used was a copper-tin alloy, as is Gem Bronze albeit with a few awesome modifications, but still a copper-tin alloy at its core.
Copper and tin have vastly different melting temperatures. copper at 1984F and tin at 450F. When ingots are cast, both today and 4000 years ago this process below will occur. The copper in the alloy will begin to cool back into a solid faster than the tin will. This results in a crystal structure that looks like a mishmash of intersecting tree-shaped objects called dendrites. For reasons we don't need to go into now, the lower melting point metals in the alloy like tin will cause the tin to freeze in a non-equilibrium state. This state looks like copper tree trunks with tin making up the leaves and the spaces that connect the trees.
When heated up the spaces between the grains (trees) can actually start to melt before the majority of the alloy is liquid. So a smith trying to forge a piece of bronze that has reached this critical temperature will cause it to crack from a single hammer blow.
Both of these issues are why Gem Bronze can't be worked under a hammer,