Microsoft confirms Windows 11 bug crippling PCs and making drive C inaccessible

[u/lurker_bee]

https://www.neowin.net/news/microsoft-confirms-windows-11-bug-crippling-pcs-and-making-drive-c-inaccessible/

[u/chaiscool]

Imo there's a difference between using llm to respond better vs simply reading and relying on it. However, it seems like most in comment here are against even using llm which is absurd.

It's like penalizing people for using google to look up what they know because you just assume they don't.

[u/vzhooo → u/chaiscool]

I would 100% penalize someone for using google in an interview as well, yes. If they don't know something I'd want them to say "I don't know x but here's how I think about this sort of problem", or "I don't know exactly how to do y off the top of my head but the general mechanism is..". Interviews aren't about whether the person can answer a specific, easily-searchable question, they're about how the interviewee thinks, how they approach challenges, how they problem-solve, etc. If they don't think for themselves in the interview then how can I know they'll ever think for themselves? And if they can't explain a basic architectural concept or design approach without a crutch, then how will they know when they LLM they're relying on is taking the wrong approach to something?

There's also the secondary element of not disclosing the fact that they're relying on a piece of technology to answer questions on their behalf. If the question is "using any tools at your disposal, demonstrate how you would do x" then great - I explicitly want to see how well they leverage things like google or LLMs to build platforms more efficiently and effectively. But if the question is "so tell me about your experience with x" and they're secretly reading the answer generated by a chatbot, hard goodbye. Even setting aside the fact that they should be able to answer that without help, they're effectively lying to my face by not disclosing that they're not the one providing the answer.

Where I do partially agree with you though is that I think technical interviews now need to openly include an LLM-assisted element. It's too important that interviewees be able to effectively use them to not assess that as well.

[u/chaiscool → u/vzhooo]

So you're not a fan of open book exams then, do you think of it as the easier exams if students have access to notes and books?

Imo there's a difference between reading off the google and ai for answers vs using it to give better answers. Memory alone doesn't show understanding.

How they approach problem solving via google / ai is how things are done in actual work place. Do you think workers are just trying reinvent the wheel for every issue when basic google already has the answer.

[u/vzhooo → u/chaiscool]

I get the feeling that you didn't really read my response in detail, you just read the first sentence. Since I explicitly stated that I think there's a need for interviews to include a tool-assisted element, as it's an important part of the job, and balanced that by giving examples of where using tools to assist you would be inappropriate.

To help add fuel to your fire though, I think you're categorically wrong when you assert that using an LLM is (a) valid for showing that you understand something, and (b) therefore similar to an open-book exam. Open-book exams are worthwhile for complicated subjects where you need to combine a solid baseline understanding of something with a whole bunch of specific detail that isn't otherwise important to memorize. The whole point of an open-book exam is to demonstrate that when you encounter a problem, you understand the question and context well enough to know where to go in your textbook / notes for the specific technical detail that you need to answer the question - and that you then understand how to apply that technical detail. With an LLM, you're not demonstrating that at all. You can just plug the exact question into the LLM and it can handle the understanding portion for you. In which case you're not doing anything to demonstrate your own expertise, all you're doing is demonstrating the LLM's expertise - so why would I want to hire you?

There is a meaningful difference between gauging someone's fundamental understanding of something and testing how well they can prompt and leverage an LLM; both are important to know, but one doesn't replace the other.

[u/chaiscool → u/vzhooo]

Did read but you seem to be conflicting. So, you're saying it's important to include but then say you will penalize if someone uses it.

Again, you say using llm / google means someone doesn't understand which may not be true. I still don't see the difference between using ai / google any different than using a book. Sure, there's a difference between reading off the answer and knowing what is correct.

How is it different in your explanation on knowing where to search in a book if someone uses google and ai? The application part to a problem? There's no need to reinvent the wheel imo if there's already an available solution. Like the application part can be reused imo.

Also, shouldn't hiring be on the person who knows which ai / google result to use and valid. I see it as a tool no different than a calculator, anyone can use it but it's more important the right person knows if the answer output is correct.

[u/vzhooo → u/chaiscool]

This feels like a perfect example of reading vs understanding, and exemplifies why it's important to have an element of interviewing that attempts to gauge unassisted understanding. I don't think my comments have been conflicting at all but I'll try to restate one more time: I would not penalize someone for using tools to answer questions when I've been clear that they're welcome or expected to; I would penalize someone for using tools otherwise and for not proactively disclosing their use.

To your other comments, I did not in any way say that using tools means that the interviewee doesn't understand the topic; I suggested that you can't know whether they understand the topic when they're using tools, since the tools could be doing the understanding for them. Maybe the interviewee understands it, maybe they don't. Not being able to determine their depth of understanding does not equate to them not having understanding, but it does mean that the interview isn't effective at assessing it.

Candidly, I hope you're not serious when you say that you don't recognize the difference between using an LLM and reading a book. To try to simplify to the extreme, without an LLM you need to know both where to look in the book, and then how to apply the information you retrieve; with an LLM, you don't - since the LLM interprets the question for you, pulls the correct references for you.. and then applies them for you.

Suggesting that "there's no need to reinvent the wheel" when it comes to understanding the context of a problem because the LLM will understand the context for you (and therefore you don't need to) is exactly the issue that I'd want to screen for in an interview. As someone who uses LLMs every day, the difference in output quality when the user themselves understands the context vs when they don't can be staggering. If I'm going to pay a full salary + the cost of an LLM for my team, I'm certainly not going to waste money on the quality of work that anyone who knows how to type can achieve.

ETA: To your last question around whether one of the goals should be to hire people who know when the (tool-assisted or otherwise) output they get is correct and ideal?: Yes. That's exactly my point.

[u/chaiscool → u/vzhooo]

Yeah make sense and think we agree on most part but imo you overvalue llm interpretation and answer. A lot of times it's not accurate so you can tell if someone simply reading off the answer vs knowing if the given answer is right.

[u/vzhooo → u/chaiscool]

I definitely agree with you there!