Skip to content

Commit 38a052a

Browse files
bjhargravebjhargrave
committed
Add a skill to check if a yes-or-no question is answerable
Signed-off-by: Alexandre Evfimievski <[email protected]> Signed-off-by: BJ Hargrave <[email protected]>
1 parent 2631714 commit 38a052a

File tree

2 files changed

+265
-0
lines changed

2 files changed

+265
-0
lines changed

‎compositional_skills/grounded/linguistics/info_extraction/question_answering/yes_no/attribution.txt

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
created_by: ae2015
2+
3+
Title of work: Yes-or-no question answerability, context of example 1 (out of 6)
4+
Link to work: N/A
5+
Revision: N/A
6+
License of the work: Apache-2.0
7+
Creator names: self-authored
8+
9+
Title of work: Yes-or-no question answerability, context of example 2 (out of 6)
10+
Link to work: N/A
11+
Revision: N/A
12+
License of the work: Apache-2.0
13+
Creator names: self-authored
14+
15+
Title of work: Yes-or-no question answerability, context of example 3 (out of 6)
16+
Link to work: N/A
17+
Revision: N/A
18+
License of the work: Apache-2.0
19+
Creator names: self-authored
20+
21+
Title of work: Yes-or-no question answerability, context of example 4 (out of 6)
22+
Link to work: https://www.dmv.ca.gov/portal/uploads/2020/06/List_of_Docs_REALID.pdf
23+
Revision: as of April 13, 2024
24+
License of the work: public domain
25+
Creator names: California Department of Motor Vehicles Authors
26+
27+
Title of work: Yes-or-no question answerability, context of example 5 (out of 6)
28+
Link to work: N/A
29+
Revision: N/A
30+
License of the work: Apache-2.0
31+
Creator names: self-authored
32+
33+
Title of work: Yes-or-no question answerability, context of example 6 (out of 6)
34+
Link to work: https://science.nasa.gov/solar-system/
35+
Revision: as of April 13, 2024
36+
License of the work: public domain
37+
Creator names: NASA Authors

‎compositional_skills/grounded/linguistics/info_extraction/question_answering/yes_no/qna.yaml

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
version: 3
2+
created_by: ae2015
3+
task_description: >
4+
Given a context document and a user's question, determine whether you have
5+
enough information to provide a yes-or-no answer to the user's question.
6+
seed_examples:
7+
- context: >
8+
Document:
9+
10+
All regular employees except those on a leave of absence (LOA) must file Form XYZ.
11+
If you are a regular employee on a LOA, you do not need to file the form. If you
12+
are a supplemental employee (type B1, type B2, or type B3), your employee type
13+
determines which form you must file, as follows:
14+
15+
* If you are a type B1 employee, you must follow the same rules that apply to the
16+
regular employees.
17+
18+
* If you are a type B2 employee, the rules that apply to you are different from
19+
those that apply to the regular employees. See Pub. 31416 to find out which forms
20+
you should file.
21+
22+
* If you are a supplemental employee of neither type B1 nor type B2, you are a
23+
type B3 employee. Different rules apply for each part of the year. For information
24+
on type B3 employees, see Pub. 31416.
25+
question: >
26+
User asks the following question:
27+
28+
"I am a supplemental employee of type B1 on a LOA. Should I file Form XYZ?"
29+
30+
As the responding agent, do you have enough information to provide a yes-or-no answer
31+
to the user's question? Answer Yes or No, then explain your answer.
32+
answer: >
33+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
34+
Specifically, the answer to the user's question is "no". Indeed, the user is a
35+
supplemental employee of type B1 and is on a LOA. According to the document,
36+
supplemental employees of type B1 must follow the same rules that apply to the
37+
regular employees, and a regular employee on a LOA does not need to file Form XYZ.
38+
Hence, the user does not need to file Form XYZ. I reached this conclusion using
39+
only the information available in the document and in the user's question.
40+
41+
- context: >
42+
Document:
43+
44+
All regular employees except those on a leave of absence (LOA) must file Form XYZ.
45+
If you are a regular employee on a LOA, you do not need to file the form. If you
46+
are a supplemental employee (type B1, type B2, or type B3), your employee type
47+
determines which form you must file, as follows:
48+
49+
* If you are a type B1 employee, you must follow the same rules that apply to the
50+
regular employees.
51+
52+
* If you are a type B2 employee, the rules that apply to you are different from
53+
those that apply to the regular employees. See Pub. 31416 to find out which forms
54+
you should file.
55+
56+
* If you are a supplemental employee of neither type B1 nor type B2, you are a
57+
type B3 employee. Different rules apply for each part of the year. For information
58+
on type B3 employees, see Pub. 31416.
59+
question: >
60+
User asks the following question:
61+
62+
"I am a supplemental employee of type B1, not on a LOA. Should I file Form XYZ?"
63+
64+
As the responding agent, do you have enough information to provide a yes-or-no answer
65+
to the user's question? Answer Yes or No, then explain your answer.
66+
answer: >
67+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
68+
Specifically, the answer to the user's question is "yes". Indeed, the user is a
69+
supplemental employee of type B1 and is not on a LOA. According to the document,
70+
supplemental employees of type B1 must follow the same rules that apply to the
71+
regular employees, and all regular employees except those on a LOA must file
72+
Form XYZ. Hence, the user must file Form XYZ. I reached this conclusion using only
73+
the information available in the document and in the user's question.
74+
75+
- context: >
76+
Document:
77+
78+
The Helping Hands Program enables the employees of XYZ Technologies to support
79+
nonprofit organizations within the communities where the employees live and work.
80+
All active, regular employees of XYZ Technologies are eligible to participate in
81+
the program. The following categories are ineligible to participate in the program:
82+
83+
* Retirees who no longer work at XYZ Technologies
84+
85+
* XYZ employees on a leave of absence
86+
87+
* Supplemental employees and contractors who are not regular XYZ employees
88+
89+
Please note that XYZ employees on a leave of absence can participate in the Helping
90+
Hands Program once they return to an active status.
91+
question: >
92+
User asks the following question:
93+
94+
"I am an active, regular employee of XYZ Technologies. I am not on a leave of absence.
95+
Can I participate in the Helping Hands Program?"
96+
97+
As the responding agent, do you have enough information to provide a yes-or-no answer
98+
to the user's question? Answer Yes or No, then explain your answer.
99+
answer: >
100+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
101+
Specifically, the answer to the user's question is "yes". Indeed, the user is an
102+
active, regular employee of XYZ Technologies. According to the document, all active,
103+
regular employees of XYZ Technologies are eligible to participate in the Helping
104+
Hands Program. The document excludes retirees, XYZ employees on a leave of absence,
105+
and those who are not regular XYZ employees; however, the user is an active and
106+
regular employee who is not on a leave of absence, so the exclusions do not apply.
107+
Therefore, the user can participate in the Helping Hands Program. I reached this
108+
conclusion using only the information available in the document and in the user's
109+
question.
110+
111+
- context: >
112+
Document:
113+
114+
Applying for a driver's license is as easy as 1-2-3! You can apply for a driver's license
115+
if you bring documents from each of the three sections below:
116+
117+
Section 1. Proof of Identity - one document is required from the list below:
118+
119+
* Unexpired passport
120+
121+
* Birth certificate
122+
123+
* Certificate of naturalization or citizenship
124+
125+
* Valid, unexpired Permanent Resident Card
126+
127+
Section 2. Proof of Social Security Number - one document is required from the list below:
128+
129+
* Social Security card
130+
131+
* W-2 form
132+
133+
* SSA-1099 form
134+
135+
* Paystub
136+
137+
Section 3. Proof of Residency - two different documents are required from the list below:
138+
139+
* Home utility bill
140+
141+
* Medical document
142+
143+
* Proof of car registration
144+
145+
* Proof of employment
146+
147+
* Proof of insurance
148+
149+
* Mortgage bill
150+
151+
* Lease agreement
152+
153+
* Tax return
154+
question: >
155+
User asks the following question:
156+
157+
"Can I apply for a driver's license if I bring my birth certificate, my W-2 form, and
158+
my proof of car registration?"
159+
160+
As the responding agent, do you have enough information to provide a yes-or-no answer
161+
to the user's question? Answer Yes or No, then explain your answer.
162+
answer: >
163+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
164+
Specifically, the answer to the user's question is "no". Indeed, the user wants to
165+
bring a birth certificate, a W-2 form, and a proof of car registration. According to
166+
the context document, to apply for a driver's license, the user must bring one proof
167+
of Identity, one proof of Social Security Number, and two different proof of Residency
168+
documents. A birth certificate is one proof of Identity, a W-2 form is one proof of
169+
Social Security Number, and a proof of car registration is one proof of Residency
170+
document. The user cannot apply for a driver's license without bringing a second proof
171+
of Residency document. I reached this conclusion using only the information available
172+
in the context document and in the user's question.
173+
174+
- context: >
175+
Document:
176+
177+
XYZ Corporation released three models of phones in 2024: Model MX, Model MY, and
178+
Model MZ. Model MX is the budget-conscious choice, it has 6-inch screen, a single
179+
rear camera, and 32 GB of storage. Model MY is the most portable choice, with a
180+
5-inch screen, a dual rear camera, and 32 GB of storage. Model MZ is the luxury
181+
choice, sporting a 7-inch screen, a dual rear camera, and 64 GB of storage.
182+
These three models are the only ones available for sale in our store.
183+
question: >
184+
User asks the following question:
185+
186+
"Does your store sell a phone made by XYZ Corporation that has a 6-inch screen and
187+
a dual rear camera?"
188+
189+
As the responding agent, do you have enough information to provide a yes-or-no answer
190+
to the user's question? Answer Yes or No, then explain your answer.
191+
answer: >
192+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
193+
Specifically, the answer to the user's question is "no". Indeed, the user is looking
194+
for a phone made by XYZ Corporation that has a 6-inch screen and a dual rear camera.
195+
According to the document, our store sells only three models of phones made by
196+
XYZ Corporation: Model MX, Model MY, and Model MZ. Only Model MX has a 6-inch screen,
197+
but it has a single rear camera; models MY and MZ do not have a 6-inch screen. Hence,
198+
our store does not sell a phone the user is looking for. I reached this conclusion
199+
using only the information available in the document and in the user's question.
200+
201+
- context: >
202+
Document:
203+
204+
Solar System Overview
205+
206+
The solar system has one star, eight planets, five dwarf planets, at least 290 moons,
207+
more than 1.3 million asteroids, and about 3,900 comets. It is located in an outer
208+
spiral arm of the Milky Way galaxy called the Orion Arm, or Orion Spur. Our solar
209+
system orbits the center of the galaxy at about 515,000 mph (828,000 kph). It takes
210+
about 230 million years to complete one orbit around the galactic center.
211+
212+
We call it the solar system because it is made up of our star, the Sun, and everything
213+
bound to it by gravity - the planets Mercury, Venus, Earth, Mars, Jupiter, Saturn,
214+
Uranus, and Neptune; dwarf planets Pluto, Ceres, Makemake, Haumea, and Eris - along
215+
with hundreds of moons; and millions of asteroids, comets, and meteoroids.
216+
question: >
217+
User asks the following question:
218+
219+
"Does the solar system have two stars?"
220+
221+
As the responding agent, do you have enough information to provide a yes-or-no answer
222+
to the user's question? Answer Yes or No, then explain your answer.
223+
answer: >
224+
Yes, I have enough information to provide a yes-or-no answer to the user's question.
225+
Specifically, the answer to the user's question is "no". According to the document,
226+
the solar system has only one star - the Sun, not two stars. I reached this
227+
conclusion using only the information available in the document and in the user's
228+
question.

0 commit comments

Comments
 (0)