Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spec clarification on Array and null #4364

Closed
scabug opened this issue Mar 19, 2011 · 7 comments
Closed

spec clarification on Array and null #4364

scabug opened this issue Mar 19, 2011 · 7 comments

Comments

@scabug
Copy link

scabug commented Mar 19, 2011

There is a comment in Arrays.scala which says:

def unapplySeq[T](x: Array[T]): Option[IndexedSeq[T]] =
  if (x == null) None else Some(x.toIndexedSeq) 
  // !!! the null check should to be necessary, but without it 2241 fails. Seems to be a bug
  // in pattern matcher.  

I think the comment contradicts the spec, and the behavior is correct. The only thing the spec says about null is that typed patterns match non-null values. In order for the code above not to NPE, we'd have to stop calling extractors when the scrutinee is null. Since that would make it impossible to write an extractor which deals with null, it seems undesirable.

@scabug
Copy link
Author

scabug commented Mar 19, 2011

Imported From: https://issues.scala-lang.org/browse/SI-4364?orig=1
Reporter: @paulp

@scabug
Copy link
Author

scabug commented Mar 22, 2011

@odersky said:
Paul, you want to suggest some spec language to fix this? Thanks

@scabug scabug added this to the Backlog milestone Apr 7, 2017
@SethTisue SethTisue changed the title spec clarification spec clarification on Array and null Feb 17, 2018
@eed3si9n eed3si9n self-assigned this Mar 5, 2018
eed3si9n added a commit to eed3si9n/scala that referenced this issue Mar 5, 2018
Ref scala/bug#4364

The specification clarification that is needed is this: What should the pattern matcher do when encountering a `null` and an extractor pattern?

As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`. See `Animal` example below.

There are potentially two things we can do.
1. Be ok with the status quo. Make sure we handle `null` in all our extractors.
2. Not be ok with the status quo. Suppose we would want extractor pattern to be non-null values only, then that's a spec change and change in pattern matcher's behavior. It might also break existing code that handled `null` in ever so subtle way.

I think Paul is suggesting option 1 in scala/bug#4364, not spec change. Given that there are no outcry of `null` handling besides scala/bug#2241, I think that's fine too.

### MyNull

```scala
scala> object MyNull {
     |   def unapply(x: AnyRef): Option[AnyRef] =
     |     if (x eq null) Some(x) else None
     | }
defined object MyNull

scala> null match { case MyNull(_) => "yes"; case _ => "no" }
res4: String = yes
```

### Animal

```scala
scala> object Animal {
     |   def unapply(x: AnyRef): Option[AnyRef] =
     |     if (x.toString == "Animal") Some(x)
     |     else None
     | }
defined object Animal

scala> null match { case Animal(_) => "yes"; case _ => "no" }
java.lang.NullPointerException
  at Animal$.unapply(<console>:13)
  ... 37 elided
```
eed3si9n added a commit to eed3si9n/scala that referenced this issue Mar 5, 2018
Ref scala/bug#4364

The specification clarification that is needed is this: What should the pattern matcher do when encountering a `null` and an extractor pattern?

As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`. See `Animal` example below.

There are potentially two things we can do.
1. Be ok with the status quo. Make sure we handle `null` in all our extractors.
2. Not be ok with the status quo. Suppose we would want extractor pattern to be non-null values only, then that's a spec change and change in pattern matcher's behavior. It might also break existing code that handled `null` in ever so subtle way.

I think Paul is suggesting option 1 in scala/bug#4364, not spec change. Given that there are no outcry of `null` handling besides scala/bug#2241, I think that's fine too.

### MyNull

```scala
scala> object MyNull {
     |   def unapply(x: AnyRef): Option[AnyRef] =
     |     if (x eq null) Some(x) else None
     | }
defined object MyNull

scala> null match { case MyNull(_) => "yes"; case _ => "no" }
res4: String = yes
```

### Animal

```scala
scala> object Animal {
     |   def unapply(x: AnyRef): Option[AnyRef] =
     |     if (x.toString == "Animal") Some(x)
     |     else None
     | }
defined object Animal

scala> null match { case Animal(_) => "yes"; case _ => "no" }
java.lang.NullPointerException
  at Animal$.unapply(<console>:13)
  ... 37 elided
```
@SethTisue
Copy link
Member

I’ve labeled as “good first issue”, on the optimistic guess that a modest spec change might be sufficient to resolve this. Might :-)

eed3si9n added a commit to eed3si9n/scala that referenced this issue Apr 1, 2018
Ref scala/bug#4364
This is a general fix for scala/bug#2241 and scala/bug#8787.

scala/bug#4364 raises the question of what should the pattern matcher do when encountering a `null` and an extractor pattern. As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`.

There are potentially two things we can do.
1. Be ok with the status quo. Clarify in SLS that all extractor authors must handle `null`.
2. Not be ok with the status quo. Change SLS and implementation such that `null` is treated as no match (empty).

I've already sent option 1 as scala#6374.
This implements option 2.
@eed3si9n
Copy link
Member

eed3si9n commented Apr 1, 2018

I've sent two competing pull requests addressing this issue.

  1. scala/bug#4364. Mention null in extractor pattern scala#6374 takes Paul's position, and treats null passing to extractor is legal, and just adds clarification to the spec that null handling is up to extractor authors if so desired.
  2. Make extractor patterns null safe scala#6485 takes Martin's position, and treats null passing to extractor as a bug, and changes SLS and the implementation such that null is automatically treated as no match (empty), which effectively generalizes previous ad-hoc null-proofings in scala/scala@d5b02c8#diff-1cef0df5926a4a2b2aa56f8f32354363R376 and SI-8787 Regextraction is null-proof scala#3923.

Not sure if this still counts as modest spec change, but I am just going where the code is taking me.

@eed3si9n eed3si9n added the has PR label Apr 1, 2018
eed3si9n added a commit to eed3si9n/scala that referenced this issue Apr 1, 2018
Ref scala/bug#4364
This is a general fix for scala/bug#2241 and scala/bug#8787.

scala/bug#4364 raises the question of what should the pattern matcher do when encountering a `null` and an extractor pattern. As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`.

There are potentially two things we can do.
1. Be ok with the status quo. Clarify in SLS that all extractor authors must handle `null`.
2. Not be ok with the status quo. Change SLS and implementation such that `null` is treated as no match (empty).

I've already sent option 1 as scala#6374.
This implements option 2.
@NthPortal
Copy link

Though allowing pattern matching on null is more flexible(-ish), I feel like it's usually not intended, and can easily cause bugs if you don't realize that you need to check for null in your extractor

eed3si9n added a commit to eed3si9n/scala that referenced this issue Apr 12, 2018
Ref scala/bug#4364
This is a general fix for scala/bug#2241 and scala/bug#8787.

scala/bug#4364 raises the question of what should the pattern matcher do when encountering a `null` and an extractor pattern. As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`.

There are potentially two things we can do.
1. Be ok with the status quo. Clarify in SLS that all extractor authors must handle `null`.
2. Not be ok with the status quo. Change SLS and implementation such that `null` is treated as no match (empty).

I've already sent option 1 as scala#6374.
This implements option 2.
eed3si9n added a commit to eed3si9n/scala that referenced this issue May 4, 2018
Ref scala/bug#4364
This is a general fix for scala/bug#2241 and scala/bug#8787.

scala/bug#4364 raises the question of what should the pattern matcher do when encountering a `null` and an extractor pattern. As it stands, SLS does not say anything about `null` for extractor pattern, which leads:
- Pattern matcher will happily pass `null` into your extractor.
- I can write a `null`-matching extractor MyNull.
- Thus, all extractor authors must check for `null` if they would like to access the passed in argument `x`.

There are potentially two things we can do.
1. Be ok with the status quo. Clarify in SLS that all extractor authors must handle `null`.
2. Not be ok with the status quo. Change SLS and implementation such that `null` is treated as no match (empty).

I've already sent option 1 as scala#6374.
This implements option 2.
adriaanm pushed a commit to eed3si9n/scala that referenced this issue Jun 1, 2018
Until now, the spec did not say anything about `null` for extractor pattern, so that:
  - The pattern matcher would happily pass `null` into your extractor.
  - One could write a `null`-matching extractor `MyNull`.
  - But all extractor authors must consider `null` as a possible argument value.

No more! The pattern matcher inserts a non-`null` check before invoking an extractor,
so that you don't have to.

This is a general fix for scala/bug#2241, scala/bug#8787. See scala/bug#4364.
eed3si9n added a commit to eed3si9n/scala that referenced this issue Jun 2, 2018
Until now, the spec did not say anything about `null` for extractor pattern, so that:
  - The pattern matcher would happily pass `null` into your extractor.
  - One could write a `null`-matching extractor `MyNull`.
  - But all extractor authors must consider `null` as a possible argument value.

No more! The pattern matcher inserts a non-`null` check before invoking an extractor,
so that you don't have to.

This is a general fix for scala/bug#2241, scala/bug#8787. See scala/bug#4364.
@Philippus
Copy link
Member

fixed in scala/scala#6485 ?

@SethTisue SethTisue modified the milestones: Backlog, 2.13.0-M5 Feb 25, 2019
@paulp
Copy link

paulp commented Feb 25, 2019

You could use an annotation e.g. @HandlesNull on an extractor to signal the compiler that null should be passed as an argument rather than swallowed as match failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants