Drupal is written in English. We have created some custom modules where the strings in the module are written in French. These are passed through the t() so can be translated.
I am a little confused how I can show my site in both French and English when the default site language English is in both language. Please could you point me in the correct direction?
When you pass string through t() without specifying a langcode, then it will default back to English. So, if string is a French word, it will be handled as an English word.
Always use English words in t() and use the translate interface to translate it to French.
As per https://api.drupal.org/api/drupal/includes!bootstrap.inc/function/t/7 set the langcode as such t($string, $args = array(), $options = array('langcode' => 'fr'))
I'm not sure from your question whether you have successfully got your site working in both English & French, just in case, have you reviewed everything at https://localize.drupal.org/translate/languages/fr
It is normal practice that if you're writing custom modules which will be used across multiple languages, the strings in the module are written in English, as you point out that Drupal is written in English, then provide a translation using po files