$(2)解:在y=-x^2+2x+3中,令y=0,得x=3或x=-1$
$∴A(-1,0)$
$∵B(3,0),C(0,3)$
$∴OB=OC,AB=4,BC=3\sqrt{2}$
$∴∠ABC=∠MFB=∠CFE=45°$
$∵C、E、F为顶点的三角形与△ABC相似$
$∴B和F为对应点$
$设E(m,-m^2+2m+3),则F(m,-m+3)$
$∴EF=(-m^2+2m+3)-(-m+3)=-m^2+3m$
$过点F作FN⊥CO于点N$
$则在Rt△CNF中$
$FN=m,CN=OC-ON=3-(-m+3)=m$
$∴CF=\sqrt{m^2+m^2}=\sqrt{2}m$
$①当△ABC∽△CFE时$
$\frac {AB}{CF}=\frac {BC}{EF}$
$∴\frac {4}{\sqrt{2}m}=\frac {3\sqrt{2}}{-m^2+3m}$
$解得m=\frac32或m=0(舍去)$
$∴EF=\frac94$
$②当△ABC∽△EFC时$
$\frac {AB}{EF}=\frac {BC}{CF}$
$∴\frac {4}{-m^2+3m}=\frac {3\sqrt{2}}{\sqrt{2}m}$
$解得m=0(舍去)或m=\frac53$
$∴EF=\frac {20}{9}$
$综上所述,EF的长度为\frac94或\frac {20}{9}$