The world of computers, it's all a box formation.
If you want a circle, you have to create a box and "cut out" corners to give it a rounded shape.
in EQ, to get any circular border, you have to create a custom bg image for it and put that as thebackground, killing the current border set on it.
Maybe since? Hard to really explain, but requires a custom graphic, modification to the EQUI_Texture.xml and EQUI_Animations.xml files and change to EQUI_PetWindow.xml
Not a "simple" change, but not that complicated.
PS: You
could get around all this by creating a custom graphic and a staticAnimation image.
Place that in the back in the Pet Window and kill the current BG (trans=true) and Border.