I've seen several spellings (Billery, Billerey, and Bellerey) in his naturalization documents (and even Bera in a US census). I'm not sure which spelling is most accurate, as both are found in France, as well as other variations of the same name. He signed his name Jean Billerey, so most likely Billerey is the correct rendition of his surname.

Based upon the information in the naturalization documentation, he most likely came from the Haute-Saóne region of France (the handwritten info on the document is spelled "Hosannes", most likely a misspelling of Haute-Saóne), a northeastern area of France near Germany.

John Bellerey
Declaration
Filed February 10, 1841

Image of Declaration
Text of Declaration [bold italics = hand-written entries]:

Declaration filed Feb 10, 1841

STATE OF NEW YORK,

JEFFERSON COUNTY, SS.

  John Billerey being duly sworn before [Daniel Lee] Peleg Burchard, Esq., Clerk of the Court of Common Pleas, of said county, saith that he was born in the Department Hosonnes in France that he is of the age of Forty Seven years; that he owes allegiance to the King of France that he emigrated from Haven de Grace in France to the town of City of New York in the State of New York, on or about the 15^th day of April 1831 that he is now residing in the town of Lyme in the State of New York, where he has settled and is intending to reside; and that it is bona fide the intention of this Deponent to become a Citizen of the United States, and to renounce forever all allegiance and fidelity to any and every foreign power, potentate, state and sovereignty whatever, and particularly to the King of France.

Jean Billerey {signature}

5471
John Billery
Naturalization Papers
1843
Filed Febr 27, 1843

Image of Naturalization Document
Text of Naturalization document [bold italics = hand-written entry]:

State of New-York,

Jefferson County, SS. John Billerey, being duly sworn, doth depose and say that he is a native of Hosannes [sic - Haute-Saóne, presumably] in France of the Kingdom of France and lately owing to the King of the French; that he is of the age of 49 years; that he came to reside in the United States in the year 1831 and has continued to reside within the jurisdiction of the United States for more than five years last past; [and has not been out of the jurisdiction of the United States neither? five year last past] and that it has been his bona fide intention to become a citizen of the United States, and to renounce and abjure all allegiance and fidelity to every foreign Prince, Power, Potentate, State or Sovereignty whatever, for mor than five years last past: and especially that of the King of the French and become a Citizen of the United States of America.

Sworn and subscribed in open Court, this
27 day of Febr 1843

Daniel Lee Clark {signature}                                                                                    Jean Billerey {signature}

STATE OF NEW-YORK,
Jefferson COUNTY, SS,
I, John Billerey, do swear that I will support the Constitution of the United States of America, and that I do absolutely and entirely renounce and abjure all allegiance and fidelity to every foreign Prince, Power, Potentate, State or Sovereignty whatsoever, and particularly to the King of the French whose subject I was. And further, that I never have borne any hereditary title, or been of any of the degrees of nobility of said Kingdom

Sworn in open Court, this 27 day
of Febr 1843                                                               Jean Billerey {signature}  Daniel Lee Clark {signature}

STATE OF NEW-YORK,
Jefferson, COUNTY, SS.
Xavier Colin and Frederick Cory? of the county aforesaid, being severally sworn, depose and say, that we are citizens of the United States, and have been acquainted with John Billerey for more than five years last past; that said John Billerey has resided within the United States for more than five years last past, and within the State of New-York 5 years; that he has behaved as a man of good moral character, attached to the principles of the Constitution of the United States, and well disposed to the good order and happiness of the same.

Sworn in open Court, this 27 day                        Xavier Colin {signature}          of Febr 1843                                                  Frederic Lory {signature}          Daniel Lee Clark {signature}