Skip to content

metric computation from DB#62

Open
raghavm243512 wants to merge 6 commits intomainfrom
pr/rm/auth_success
Open

metric computation from DB#62
raghavm243512 wants to merge 6 commits intomainfrom
pr/rm/auth_success

Conversation

@raghavm243512
Copy link
Copy Markdown
Collaborator

@raghavm243512 raghavm243512 commented Apr 17, 2026

Marks successful auth in the DB under "session" to make consistent and usable for future domains
Auth success it considered valid if final DB["session"] value is a superset of expectation (minimum conditions)
Airline expected final DB is changed to include a session value as baseline for auth
All hashes will pop the session key before computation (so no impact to task completion or logging/saving of files)

@raghavm243512 raghavm243512 marked this pull request as ready for review April 20, 2026 21:22
@tara-servicenow
Copy link
Copy Markdown
Collaborator

tara-servicenow commented Apr 21, 2026

One small thing - I do want authentication success to be a prerequisite for task_completion. We can modify the code for task_completion so that the requirement is now database hash equivalence and also authentication success. Otherwise this looks great!

@tara-servicenow
Copy link
Copy Markdown
Collaborator

One more issue here: in the airline dataset, there are a few adversarial records where the user refuses to provide the confirmation code and this is expected. In these cases, we don't authentication to happen so session in the expected scenario DB should be an empty dict. But i see in the dataset you've added auth to be expected for every single record. Please check the adversarial ones and change it to {} when we don't expect authentication to happen (for example 7.1.1 and 7.4.1 and maybe others)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants