Week 9 - 12 - Facial Recognition to Crysis

Unfortunately due to the fact that I needed my facial recognition to be in C++ and not in C#, I needed to convert my C# code to C++. After some reading I found that converting the languages is possible - though difficult due to differences in approach, the framework calls require porting to different libraries, and is often not a good candidate for a direct translation. Therefore I decided to start re-writing my code in C++ but unfortunately I still encountered problems and because I am very new to coding it tok me a very long time to understand the syntax and not get confused between both the languages. I then decided to start fresh and start another facial recognition code in C++. I found this tutorial online at http://www.shervinemami.info/faceRecognition.html.

The facial recognition according to his code runs as below:

1. Grab a frame from the camera 

// Grab the next camera frame. Waits until the next frame is ready, and
// provides direct access to it, so do NOT modify or free the returned image!
// Will automatically initialize the camera on the first frame.
IplImage* getCameraFrame(CvCapture* &camera)
{
 IplImage *frame;
 int w, h;

 // If the camera hasn't been initialized, then open it.
 if (!camera) {
  printf("Acessing the camera ...\n");
  camera = cvCreateCameraCapture( 0 );
  if (!camera) {
   printf("Couldn't access the camera.\n");
   exit(1);
  }
  // Try to set the camera resolution to 320 x 240.
  cvSetCaptureProperty(camera, CV_CAP_PROP_FRAME_WIDTH, 320);
  cvSetCaptureProperty(camera, CV_CAP_PROP_FRAME_HEIGHT, 240);
  // Get the first frame, to make sure the camera is initialized.
  frame = cvQueryFrame( camera );
  if (frame) {
   w = frame->width;
   h = frame->height;
   printf("Got the camera at %dx%d resolution.\n", w, h);
  }
  // Wait a little, so that the camera can auto-adjust its brightness.
  Sleep(1000); // (in milliseconds)
 }

 // Wait until the next camera frame is ready, then grab it.
 frame = cvQueryFrame( camera );
 if (!frame) {
  printf("Couldn't grab a camera frame.\n");
  exit(1);
 }
 return frame;
}

2. Convert the colour frame to greyscale

 // If the image is color, use a greyscale copy of the image.
 detectImg = (IplImage*)inputImg;
 if (inputImg->nChannels > 1) {
  size = cvSize(inputImg->width, inputImg->height);
  greyImg = cvCreateImage(size, IPL_DEPTH_8U, 1 );
  cvCvtColor( inputImg, greyImg, CV_BGR2GRAY );
  detectImg = greyImg; // Use the greyscale image.
 }

3. Detect a face within the greyscale camera frame

// Perform face detection on the input image, using the given Haar Cascade.
// Returns a rectangle for the detected region in the given image.
CvRect detectFaceInImage(IplImage *inputImg, CvHaarClassifierCascade* cascade)
{
 // Smallest face size.
 CvSize minFeatureSize = cvSize(20, 20);
 // Only search for 1 face.
 int flags = CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH;
 // How detailed should the search be.
 float search_scale_factor = 1.1f;
 IplImage *detectImg;
 IplImage *greyImg = 0;
 CvMemStorage* storage;
 CvRect rc;
 double t;
 CvSeq* rects;
 CvSize size;
 int i, ms, nFaces;

 storage = cvCreateMemStorage(0);
 cvClearMemStorage( storage );


 // Detect all the faces in the greyscale image.
 t = (double)cvGetTickCount();
 rects = cvHaarDetectObjects( detectImg, cascade, storage,
   search_scale_factor, 3, flags, minFeatureSize);
 t = (double)cvGetTickCount() - t;
 ms = cvRound( t / ((double)cvGetTickFrequency() * 1000.0) );
 nFaces = rects->total;
 printf("Face Detection took %d ms and found %d objects\n", ms, nFaces);

 // Get the first detected face (the biggest).
 if (nFaces > 0)
  rc = *(CvRect*)cvGetSeqElem( rects, 0 );
 else
  rc = cvRect(-1,-1,-1,-1); // Couldn't find the face.

 if (greyImg)
  cvReleaseImage( &greyImg );
 cvReleaseMemStorage( &storage );
 //cvReleaseHaarClassifierCascade( &cascade );

 return rc; // Return the biggest face found, or (-1,-1,-1,-1).

4. Crop the frame to just show the facial recognition using cvSetImageROI() and cvCopyImage().

5. Pre-process the face image

// Either convert the image to greyscale, or use the existing greyscale image.
IplImage *imageGrey;
if (imageSrc->nChannels == 3) {
 imageGrey = cvCreateImage( cvGetSize(imageSrc), IPL_DEPTH_8U, 1 );
 // Convert from RGB (actually it is BGR) to Greyscale.
 cvCvtColor( imageSrc, imageGrey, CV_BGR2GRAY );
}
else {
 // Just use the input image, since it is already Greyscale.
 imageGrey = imageSrc;
}

// Resize the image to be a consistent size, even if the aspect ratio changes.
IplImage *imageProcessed;
imageProcessed = cvCreateImage(cvSize(width, height), IPL_DEPTH_8U, 1);
// Make the image a fixed size.
// CV_INTER_CUBIC or CV_INTER_LINEAR is good for enlarging, and
// CV_INTER_AREA is good for shrinking / decimation, but bad at enlarging.
cvResize(imageGrey, imageProcessed, CV_INTER_LINEAR);

// Give the image a standard brightness and contrast.
cvEqualizeHist(imageProcessed, imageProcessed);

.....  Use 'imageProcessed' for Face Recognition ....

if (imageGrey)
 cvReleaseImage(&imageGrey);
if (imageProcessed)
 cvReleaseImage(&imageProcessed);

6. Recognise the person in the image

After talking to Steve he told me to use a TCP Server to connect my output from my facial recognition to CryEngine. He told me to do the following to link his TCP Server code to the facial recognition:

"Your main() loop from the C++ should go into the Update() function. Any variables declared outside the main() loop should become member variables (ie, declared in the .h file, then optionally instantiated in the constructor in the .cpp file). Any #include lines you have in your face detection code should go in the .h file, *below* the existing #include lines." Steve

Unfortunately I didn't have a .h file and everything was in my .cpp. As for my main() loop it was unfortunately incorrect as the actual facial recognition wasn't in that loop. I went to Steve to fix up this problem and he said the easiest solution was to use the TCP Server and use the old C# code. Please follow the link below to see Facial recognition code using a TCP Server to Crysis.

http://januarch1392.blogspot.com.au/2013/06/4.html