The Best Teachers Lie

10 Sep 2023

ChatGPT is often confidently wrong, which makes it an excellent teacher.

The brain stores what it expects to need, so manipulating new information is the best way to retain it. This is why we have workbooks as well as lectures.

But there's a more interesting solution: I can't trust anything ChatGPT says, which means that I can't take its explanations as gospel, the way I might a textbook.

Instead, I have to critically analyse each point, checking it against what I already know and the rest of the answer, in order to spot potential errors. I'm forced to immediately and deeply engage with the material.

Then I frame a followup question - either "So, this means X", or "Hangon, is that right? It conflicts with Y". Occasionally ChatGPT will double down, but mostly it will immediately correct its error. It's quite satisfying; calling out the error is like taking a pop quiz and passing.

I often use other sources in parallel, so in the case where ChatGPT is consistently hallucinating, I catch nearly all the errors. Scanning a source to check a belief is much faster than learning from scratch.

I'm sure some errors get through, at least temporarily, but since the things I'm trying to learn - music theory, low-voltage electronics, coding problems - aren't life-critical, for me this is a good tradeoff. The greater risk is not accidentally learning false facts, but failing to absorb the true ones.

Examples

ChatGPT: ... Given the range V_GS(th): Minimum: -2.0V and Maximum: -4.0V. If you consider the worst-case scenario (always a good practice in electronics design), you would assume that the gate threshold is as high as -2.0V. This means the MOSFET could start turning on at just -2.0V. ...

Me: Hangon, surely getting one with -2.0V is my best-case scenario (partially turns on, but not completely), and getting one with -4.0V is my worst-case scenario (doesn't work at all)?

ChatGPT: Yes, you are correct. I apologize for the oversight. ... The worst-case scenario would indeed be when the gate threshold is -4.0V. In this case, if you get a MOSFET with this threshold, applying a 2.2V difference at the gate would not turn on the MOSFET sufficiently.


Me: ... It needs to carry 5V at 200mA. I think I can do this using a 555 timer. I'm looking for the simplest possible solution.

ChatGPT: [full circuit design including a relay and diode]

Me:: Given that we only need to carry 200mA, can we dispense with the relay and power the sink directly through the 555?

ChatGPT: Yes, the 555 timer can source or sink up to ≈200mA. So ... you can indeed power it directly through the 555 timer without using a relay. [circuit details with relay removed]

Me:: Why do we need the diode?

ChatGPT: I apologize for the confusion earlier. ... in the described setup, the diode is redundant.


ChatGPT: ...

  const escapedBasePath = escapeForRegex(RECORDING_BASE_PATH);
  const pattern = ffmpegPath + '.*' + escapedBasePath; // match ffmpeg processes with RECORDING_BASE_PATH in their arguments
  exec(\`pkill -f "${pattern}"\`, (error) => {

...

Me: Hangon, if we're matching a substring rather than a regex, why do we have `'.*'`?

ChatGPT: You're right; I apologize for the oversight. If we're using pkill to match a substring in the full command line, there's no need to use regular expression-like syntax. ...


ChatGPT: [explaining UDP hole-punching]

Me: Does this break, if devices Y and Z both try to use it for the same destination address at the same time?

ChatGPT: If devices Y and Z behind the same NAT both attempt to initiate communication with the same external destination address (and port) simultaneously, it could indeed pose challenges for the NAT's ability to distinguish and route the returning packets correctly.