The data in itself is harmless. How the algo is programmed, who has access to it, what the intentions will be of those who can make life changing calls based on it is the dangerous still-unanswered questions society must place on the use of technology. Do people have the right to privacy? Where does this infringe on the right to privacy.

The Fourth Amendment of the United States Constitution states that:

“The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.”

A National Institute of Standards and Technology report found age, race and gender could impact the accuracy in some commonly-used facial recognition algorithms.

Source: Lawmaker Asks DHS Secretary to Reassess Facial Recognition Programs After NIST Report